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EXECUTIVE  SUMMARY 


This  study  describes  the  use  and  value  of  the  major  information 
products  and  services  provided  by  the  Defense  Technical  Information  Center 
(OTIC) .  Hie  products  and  services  considered  include  technical  report 
distribution  on  an  on  demand-basis  and  through  the  Automatic  Document 
Distribution  (ADD)  program;  secondary  information  dissemination  through 
online  searching  of  the  Technical  Reports  (TR)  data  base,  Current  Awareness 
Bibliographies  (CAB) ,  and  Technical  Abstract  Bulletins  (TAB) ;  and  provision 
of  management  information  from  three  management  data  banks.  Hie  amount  and 
kinds  of  use  of  each  of  these  products  and  services  is  addressed,  as  is  the 
value  associated  with  use. 

About  1 .1  million  copies  of  Department  of  Defense  (DoD)  technical 
reports  were  distributed  by  OTIC  in  1982.  The  estimated  157,000  scientists 
and  engineers  engaged  in  DoD  research  made  about  12.4  million  readings  of 
DoD  reports,  or  about  80  each  on  the  average.  Most  readings  were  for 
multiple  purposes,  with  the  most  common  purposes  being  application  of  the 
report  findings  to  a  current  project  and  professional  development,  current 
awareness  or  general  interest.  On  the  average,  the  amount  of  time  spent 
reading  a  DoD  technical  report  is  about  100  minutes. 

Technical  reports  read  are  identified  through  various  channels, 
including  online  searching  of  the  TR  data  base,  use  of  CAB,  and  use  of  TAB. 
Nearly  half  of  all  DoD  reports  read  are  identified  through  an  online 
search,  with  about  600,000  such  searches  conducted  in  1982.  About  12 
percent  and  6  percent  of  all  readings  came  through  use  of  TAB  and  CAB, 
respectively.  There  are  about  25  report  readings  for  every  biweekly  copy 
of  TAB  issued,  and  about  14  readings  for  every  CAB  copy  distributed. 

OTIC's  management  data  bases  include  the  R&D  Work  Ohit  Information 
System  (WUIS) ,  the  R&D  Program  Planning  Data  Base  (R&DPP) ,  and  the  Inde¬ 
pendent  Research  and  Development  Data  Base  (IR&D) .  Uses  of  the  three  data 


bases  in  1982  are  estimated  at  280,000,  with  about  three-quarters  of  the 
use  being  of  WUIS.  There  were  about  14,000  users  of  the  management  data 
banks  in  1982,  with  an  average  of  about  20  uses  each. 

The  value  of  OTIC's  information  products  and  services  is  considered 
from  two  perspectives:  that  of  the  user,  expressed  in  terms  of  willingness 
to  pay,  and  that  of  the  funders,  expressed  in  terms  of  savings  resulting 
from  reading  of  DoD  technical  reports.  Value  is  estimated  at  $367  million 
from  the  users'  perspective.  From  the  funders'  perspective,  value  is 
estimated  at  $37.5  billion,  100  times  the  value  expressed  in  terms  of 
willingness  to  pay. 

to  consider  the  value  of  the  DHC  products  and  services  relative  to 
the  value  of  substitute  services,  estimates  were  made  of  the  report  read¬ 
ings  that  would  be  lost  if  information  expenditures  were  fixed  and  other 
available  bibliographic  services  were  used  to  identify  DoD  reports.  The 
value  of  readings  lost  by  use  of  such  substitutes  is  taken  as  the  value  of 
the  DTIC  products  and  services.  The  total  value  to  the  user  measured  in 
this  way  is  $77  million  and  the  value  to  the  funder  is  $8.3  billion,  again 
approximately  a  hundredfold  increase. 

These  findings  establish  significant  levels  of  value  associated 
with  the  provision  of  OTIC  products  and  services.  There  are  substantial 
readings  of  DoD  technical  reports,  with  about  12.4  million  such  readings 
estimated  for  1982.  The  savings  value  associated  with  an  individual  read¬ 
ing  —  $4,600  on  the  average  —  is  high.  The  total  value  to  the  Department 
of  Defense,  reflecting  only  those  readings  coming  through  OTIC  biblio¬ 
graphic  products,  is  $37.5  billion.  This  suggests  a  considerable  return  cm 
the  investment  in  OTIC's  information  products  and  services. 


CHAPTER  1 


INTRODUCTION 


Hie  value  of  information  services  and  systems  to  the  user  conmuni- 
ties  they  are  intended  to  serve  is  receiving  considerable  attention  in 
today's  clinate  of  resource  control  and  economic  rationalization .  Inplicit 
in  the  Federal  Government's  provision  of  information  products  and  services 
to  researchers  is  the  assumption  that  the  use  of  these  products  and 
services  will  contribute,  directly  or  indirectly,  to  the  Government's 
mission.  Questions  arise,  however,  in  attempting  to  pinpoint  the  linkages 
between  a  carpi ex  information  system  and  research  activities  and  in  deter¬ 
mining  the  appropriate  types  and  levels  of  services  to  be  provided.  Other 
questions  which  arise  concern  how  and  by  whom  information  services  can  best 
be  made  available. 

Among  the  agencies  facing  such  questions  is  the  Defense 
Department's  Technical  Information  Center  (OTIC).  OTIC  provides  informa¬ 
tion  cm  Department  of  Defense  (DoD)  research  and  development  activities  to 
the  Government  and  to  DoD  contractors,  subcontractors,  and  grantees.  A 
range  of  products  and  services  are  provided,  including  both  primary  and 
secondary  information  distributed  in  a  number  of  forms.  In  providing  these 
products  and  services,  DTIC  is  concerned  with  the  sorts  of  issues  mentioned 
above  and  with  the  underlying  question  of  benefits  derived  by  their  users. 
Reflecting  these  concerns,  OTIC  requested  that  King  Research,  Inc.  perform 
a  study  to  explore  the  value  of  their  products  and  services.  This  report 
documents  that  study. 

The  purpose  of  the  study  was  to  look  at  the  najor  DTIC  products  and 
services,  to  investigate  their  levels  of  use  by  DoD  scientists  and 
engineers,  and  to  estinrate  the  value  associated  with  that  use.  The  metho¬ 
dology  used  here  has  been  developed  by  King  Research  to  consider  questions 
of  value  of  information  products  and  services  of  all  types.  The  approach 
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I  taken  is  based  on  the  premise  that  the  value  of  an  information  product  or 

service  stems  from  its  use,  and  from  the  subsequent  application  of  informa¬ 
tion  acquired  from  the  product  or  serv^e.  Value  is  addressed  at  three 
levels,  or  from  three  perspectives:  that  of  the  user,  the  funder  (in  this 
case  the  Department  of  Defense),  and  of  society  in  general.  Economic 
models  are  used  to  estimate  both  the  total  value  of  infornation  provided 
through  a  particular  source  and  the  net  value  associated  with  providing  the 
information  through  one  particular  source  rather  than  some  other . 

This  approach  to  the  measurement  of  value  was  first  applied  to  the 
Department  of  Energy's  Technical  Information  Center  (TIC) .  Hie  results  of 
that  study  are  documented  in  King  Research's  Value  of  the  Ehergy  Data  Base 
[1] .  Like  OTIC,  TIC  provides  both  primary  and  secondary  information  on 
research  and  development  activities  to  scientists,  engineers  and  others 
associated  with  their  agency's  programs.  Where  appropriate  in  this  report, 
results  from  the  TIC  study  are  presented  for  comparison  with  those  from 
OTIC. 

To  obtain  information  on  the  use  and  value  of  OTIC's  products  and 
services,  existing  data  on  distribution  and  on  users  were  first  analyzed. 
From  this,  the  composition  of  the  user  community  was  identified.  Regis¬ 
tered  OTIC  users  are  generally  organizations,  representing  groups  of  indi¬ 
vidual  users.  Actual  use  data  came  from  a  telephone  survey  of  these  indi¬ 
viduals  in  the  user  organizations,  with  the  individuals  randomly  selected 
from  among  research  scientists  and  engineers  in  the  organization.  Survey 
topics  included  general  information  use,  use  of  OTIC  products  and  services, 
characteristics  of  the  last  use  of  a  particular  DTIC  product  or  service, 
and  the  value  associated  with  that  use.  More  infornation  on  survey  methods 
is  provided  in  Appendix  B. 

In  this  study  of  DTIC's  information  products  and  services,  we 
looked  closely  at  the  use  of  information  at  various  classification  levels; 
that  is,  classified,  unclassifie<yiimited,  and  unclassified/unlimited.  Hie 
purpose  of  this  was  to  look  at  differential  patterns  of  use  and  of  value  of 
each  of  these  categories  of  material.  Also  of  special  interest  in  this 
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study  was  the  use  made  of  DTIC's  management  information  data  banks,  There 
are  three  such  data  banks;  the  Research  and  Technology  Work  Unit  Informa¬ 
tion  System  (WUIS) ,  the  Research  and  Development  Program  Planning  (R&DPP)! , 
and  the  Independent  Research  and  Development  (IR&D) .  For  each  of  these 
data  baiks  we  looked  again  at  levels  of  use  and  associated  levels  of  value. 


1.2  Value  and  Its  Measurement 

The  Value  of  Information  and  Information  Products  and  Services 

Through  the  years  our  society  has  taken  information  for  granted, 
just  as  it  has  many  of  our  other  abundant  resources.  Now  with  tremendous 
pressures  being  brought  to  bear  on  budgets  and  other  resources,  the  infor- 
rration  conrunity  is  faced  with  increased  demands  to  justify  its  expendi¬ 
tures  and,  in  some  instances,  even  its  existence.  Unfortunately  there  has 
been  no  calculus  or  economic  theory  developed  to  measure  the  value  of 
information.  This  is  partially  because  in  the  past  there  has  been  no 
pressing  need  to  do  so  and  partially  because  of  the  difficulty  in  doing  so. 
The  difficulty  arises  largely  because  of  the  inability  to  define  or  quan¬ 
tify  information  and  to  separate  information  from  the  product  or  service 
used  to  transmit  it.  In  addition,  the  economic  properties  of  the 
information  (e.g.,  as  a  public  good  or  private  good)  change  depending  on 
where  in  the  information  transfer  process  it  resides  (i.e.,  in  the  hands  of 
an  author,  publisher,  library  or  user). 

Information  generated  through  public  funding,  such  as  that  produced 
through  Department  of  Defense  research  and  development  funding,  presents  an 
additional  set  of  issues.  On  the  one  hand  information  is  part  of  what  is 
produced  through  R&D  efforts  and  the  information 's  value  is  partially 
defined  by  how  much  it  is  subsequently  used.  On  the  other  hand,  R&D 
efforts  are  heavily  dependent  on  existing  knowledge  to  build  upon  or  avoid 
the  necessity  of  regenerating  existing  information.  Conducting  research 

1  R&DPP  data  bank  activity  was  discontinued  in  December  1982. 
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and  development  is  like  manufacturing  a  product  in  the  sense  that  it  is 
dependent  on  a  number  of  naterials  or  ingredients.  Infornation  is  among 
the  material  or  ingredients  that  contribute  to  research  and  development 
output.  The  purpose  of  the  Defense  Technical  Infornation  Service  is  to 
collect,  process  and  provide  access  to  information  generated  from  DoD 
research  and  development  efforts.  The  purpose  of  this  report  is  to  gain 
some  notion  as  to  the  value  of  that  infornation  and  the  contribution  that 
OTIC  makes  in  providing  it. 

As  will  be  shown  in  the  next  section,  scientists  and  engineers 
devote  an  enormous  amount  of  time  to  acquiring,  reading  and  using  informa¬ 
tion.  The  question  becomes  one  of  whether  one  can  place  a  value  on  the  use 
of  information.  King  Research  has  attempted  to  look  at  the  value  of  infor¬ 
mation  from  several  perspectives;  those  of  users  (i.e.,  the  DoD  funded 
scientists  and  engineers) ,  the  DoD  funders  and  society.  The  time  that 
scientists  and  engineers  have  to  conduct  their  work  is  a  scarce  resource 
since  there  is  so  much  demand  on  it.  Presumably,  they  make  rational  deci¬ 
sions  on  how  to  spend  that  time  in  a  way  that  is  most  beneficial  to  their 
work  and  their  own  self-interest.  They  (or  their  representatives  such  as 
librarians)  also  have  limited  funds  available  for  purchasing  information 
products  or  services  so  that  decisions  to  acquire  these  should  also  be  made 
on  a  rational  basis.  Fran  the  perspective  of  scientists  and  engineers,  one 
way  to  measure  value  is  by  measuring  what  they  are  willing  to  pay  in  terms 
of  the  moneys  exchanged  or  time  and  effort  needed  to  identify,  locate, 
order,  receive  and  use  information.  What  they  pay  in  these  ways  is  called 
effective  price  in  this  report. 

The  value  of  a  body  of  information  such  as  that  found  in  DoD 
reports  can  be  measured  by  multiplying  the  average  effective  price  times 
the  number  of  times  information  is  read  (used) .  This  might  be  considered  a 
lower  bound  of  value  represented  by  what  users  are  willing  to  pay.  It  is 
an  underestimate  in  the  sense  that  sane  users  would  presunably  be  willing 
to  pay  more.  Furthermore,  this  lower  bound  of  value  does  not  include  the 
value  derived  from  reading  infornation.  It  is  shown  in  this  report  that 
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reading  is  done  largely  for  the  purposes  of  applying  the  results  for 
conducting  research,  educating  oneself,  educating  others,  writing, 
management,  etc.  Each  of  these  purposes  should  yield  scare  benefits,  such 
as  saving  research  time  and  materials.  Thus,  additional  value  should  be 
achieved  through  improved  research  results,  education,  writing,  and 
management . 

Considered  from  the  perspective  of  scientists  and  engineers,  value 
is  measured  in  terms  of  the  user's  willingness  to  pay  as  described  above. 
Another  perspective  is  hew  the  reading  and  use  of  information  affect  their 
work.  Manifestations  of  the  value  of  reading  information  are  in  the  labor 
time  saved;  materials,  equipment,  travel  supplies,  telephone  calls,  etc. 
saved;  improved  research  and  development  activities;  improved  education  of 
oneself  and  others;  improved  decision  making;  and  so  on.  This  might  be 
thought  of  as  the  funder's  perspective.  A  third  perspective  involves  how 
the  work  of  scientists  or  engineers  affects  the  objectives  of  their 
organisations  or  the  goals  and  mission  of  society.  Examples  of  such  value 
might  be  how  the  work  results  in  better  national  defense,  improve  balance 
of  trade,  improve  quality  of  life,  educate  future  scientists  and  engineers, 
improve  the  research  and  technology  in  other  disciplines  (through 
technology  transfer),  protection  of  sensitive  information,  and  so  on.  The 
degree  of  difficulty  in  quantifying  and  measuring  value  is  least  with  the 
scientist's  or  engineer's  perspective,  greater  with  the  funder's 
perspective,  and  very  difficult  with  society's  perspective. 

All  of  these  measures  of  value  presumably  increase  with  greater 
frequency  of  use  of  the  information.  Thus,  factors  that  led  to  increased 
use  of  information  enhance  the  value  of  the  information.  Such  factors 
include  lowering  the  effective  price  of  information  which  includes  reducing 
the  charge  for  information  products  or  services  as  well  as  reducing  the 
cost  to  the  user  of  employing  them;  providing  better  information  products 
or  services;  increasing  awareness  of  the  information  and  means  of  getting 
it;  or  eliminating  constraints  on  availability  of  information  (i.e., 
security  classification,  transborder  flow,  restrictions  through  ownership 
of  intellectual  property,  etc.). 


There  are  other  factors  as  well  which  affect  the  value  of 
information.  For  example,  primary  information  is  used  only  if  the  products 
or  services  employed  to  transfer  it  are  purchased  (or  otherwise  acquired) 
and  used.  The  extent  to  which  journals  or  technical  reports,  for  exaxtple, 
are  purchased  depends  on  a  number  of  factors,  such  as  price;  quality  of 
information  content,  graphics,  format,  etc.;  performance,  such  as  speed  of 
delivery  and  currency;  and  awareness  of  the  existence  of  the  journal 
articles  and  technical  reports.  Awareness  can  come  in  many  ways,  such  as 
advertising,  word  of  mouth,  citations,  just  browsing,  and  searching 
bibliographic  products  and  services,  to  name  a  few.  Similarly,  use  of 
secondary  information  comes  only  after  purchase  or  acquisition  of  secondary 
products  and  services,  which  in  turn  depends  on  price,  performance, 
quality,  and  awareness. 

Hie  relationships  above  are  shewn  in  Figure  1,  where  statements  of 
relationships  are  positive  ones  for  illustrative  purposes.  For  example, 
lowering  effective  price,  increasing  quality,  improving  performance  or 
increasing  awareness  of  a  product  or  service  should  increase  purchases  (or 
acquisitions) .  Of  course,  subscribing  to  a  journal  does  not  necessarily 
mean  it  will  be  read.  However,  considering  all  subscribers  one  would 
expect  an  increased  number  of  subscriptions  will  yield  an  increased  number 
of  readings  and  use  of  primary  information.  Information  is  used  for  many 
purposes  such  as  education  of  oneself  and  others,  research,  and  management. 
Presumably  each  activity  will  be  improved  through  greater  use  and  ulti¬ 
mately  will  help  the  nation  by  improving  productivity,  balance  of  trade, 
use  of  resources,  quality  of  life,  and  so  on. 

Information  is  found  in  several  kinds  of  documents,  such  as  journal 
articles,  technical  reports,  books,  patent  documents,  numeric  data  bases, 
etc.  In  this  study  the  focus  is  on  the  information  found  in  technical 
reports.  An  estimate  is  made  of  the  effective  price  and  amount  of  reading 
doie  by  DoD  funded  scientists  and  engineers.  The  effective  price  was 
estimated  by  computing  user  expenditures  related  to  searching  (i.e., 
through  on-line  search,  use  of  published  bibliographies,  browsing,  etc.), 
acquiring  the  technical  reports  (i.e.,  through  personal  subscriptions  or 


Figure  1.  Factors  Related  to  Increasing  Value  of  Information 
Source:  King  Research,  Inc. 
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purchases,  libraries,  colleagues,  etc.),  and  reading.  The  expenditures 
include  all  the  labor  time  involved,  prices  paid,  and  costs  of  iraterials 
and  equipment  employed.  These  cost  data  were  found  through  secondary 
sources  (i.e.,  previous  studies  performed  by  KRI  and  others)  and  through  a 
user  survey.  The  extent  of  reading  of  journal  articles  and  technical 
reports  was  estimated  from  surveys  where  amount  of  reading  is  associated 
with  the  varous  methods  of  search  and  acquisition. 

Hie  determination  of  value  of  a  specific  information  product  or 
service  was  found  directly  and  then  by  estimating  the  cost  of  substitute 
products  or  services.  For  example,  the  value  of  TAB  was  estimated  by 
determining  what  the  cost  and  likelihood  are  of  various  substitute  methods 
of  searching  for  technical  reports.  Estiirates  are  made  for  the  cost  of 
searching  by  means  of  substitute  search  methods  where  the  least  expensive 
substitute  is  always  assumed  to  be  enployed  by  users.  A  further  assumption 
is  made  that  the  total  cost  of  searching  by  all  users  will  remain  about  the 
same  regardless  of  whether  TAB  or  substitute  search  methods  are  used.  Thus, 
since  the  substitute  search  methods  cost  more,  fewer  searches  would  be 
performed  and  fewer  readings  would  take  place.  The  number  of  searches  that 
can  be  performed  under  the  substitute  methods  at  the  total  cost  is  found  by 
dividing  the  total  search  costs  by  the  average  cost  of  searching  by  the 
substitute  methods.  The  number  of  searches  lost  and,  hence,  readings 
denied  to  users  by  not  having  TAB  is  computed. 

The  number  of  readings  per  search  is  estimated  by  observation. 
Thus,  if  fewer  searches  are  performed,  one  can  calculate  the  comparable 
number  of  readings  lost  because  a  substitute  search  method  was  employed. 
From  this,  one  can  estimate  value  computed  by  the  change  in  user's  willing¬ 
ness  (necessity)  to  pay  higher  effective  prices.  Readers  were  also  asked 
to  indicate  whether  a  specific  reading  of  an  article  or  technical  report 
yielded  a  savings  in  labor  time  (of  any  of  the  staff),  equipment,  or  sup¬ 
plies.  This  estimate  of  savings  is,  in  effect,  an  indication  of  improved 
productivity  yielded  by  reading.  From  these  estimates,  the  total  amount  of 
savings  lost  because  substitute  search  services  would  be  enployed  is  com¬ 
puted.  These  estimates  provide  a  conservative  estimate  of  the  value 
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derived  from  savings  yielded  by  readings  resulting  from  TAB  searches  per¬ 
formed  under  existing  prices.  It  is  a  conservative  estimate  because  other 
benefits  must  also  accrue.  This  analysis  was  performed  for  all 
bibliographic  services  provided  by  DTIC  and  for  the  collective  services. 

1.3  Overview  of  the  Report 


This  report  addresses  the  distribution,  use,  and  value  of  six  of 
OTIC's  information  products  and  services:  on  demand  technical  report 
distribution,  automatic  technical  report  distribution  (ADD),  online  Tech¬ 
nical  Reports  (TR)  data  bank  searches,  the  Technical  Abstract  Bulletin 
(TAB)  and  TAB  indexes.  Current  Awareness  Bibliographies  (CAB),  and 
management  information  data  bank  searches.  Chapter  2  describes  these 
products  and  services  and  their  interrelationships  and  indicates  the  level 
of  distribution  of  each  in  1982.  In  Chapter  3,  on  OTIC  use,  the  users 
surveyed  as  a  part  of  the  study  and  their  general  information  use  habits 
are  described.  Chapter  3  then  covers  usage  of  the  specific  OTIC  products 
and  services,  including  amount  of  use,  purpose  of  use,  methods  of 
identification  and  access,  and  usage  per  copy  distributed.  The  value  of 
OTIC  products  and  services,  from  the  perspective  of  users  and  funders,  is 
described  in  Chapter  4.  Fi 


Hie  Appendices  to  this  report  provide  additional  information  on  the 
study.  Appendix  B  describes  the  methods  used  in  surveying  DoD  users,  and 
Appendix  C  is  the  survey  instruments.  Appendix  A  gives  verbatim  responses 
received  from  users  in  response  to  questions  about  how  their  time  was  saved 
using  DTIC  products  and  other  issues. 


CHAPTER  2 


OTIC'S  PRODUCTS  AND  SERVICES 


The  Defense  Technical  Information  Canter  (DTIC)  is  the  clearing¬ 
house  for  the  Defense  Department's  collections  of  research  and  development 
in  virtually  all  fields  of  science  and  technology,  involving  subject 
categories  ranging  from  Aeronautics  to  Zoology.  OTIC,  a  primary  level 
field  activity  of  the  Defense  Logistics  Agency,  has  the  mission  to  exploit 
the  contents  of  its  collection  to  answer  three  basic  questions  related  to 
the  Research,  Development,  Test,  and  Evaluation  (ROT&E)  program  of  the 
Department  of  Defense  (DoD) .  These  questions  are:  (1)  What  research  is 
being  planned?  (2)  What  research  is  currently  being  performed?  and  (3) 
What  results  were  realized  by  completed  research? 

DTIC  obtains  its  information  from  Defense  and  associated  contractor 
researchers  who  are  required  to  deposit  information  (both  unclassified  and 
classified  including  Secret  and  Restricted  Data)  into  DTIC  data  bases  for 
the  subsequent  retrieval  for  eligible  users.  There  are  four  data  bases  in 
OTIC,  of  which  three  are  considered  management  information  programs.  The 
three  data  bases  of  greatest  importance  to  the  fulfillment  of  OTlC's  mis¬ 
sion  are  the  Research  and  Development  Program  Planning  (R&DPP)  consisting 
of  planned  research;  the  Research  and  Technology  Work  Unit  Information 
System  (WUIS)  containing  the  current  research  being  performed;  and  the 
Technical  Report  (TR)  Program  which  is  a  formally  documented  collection  of 
completed  research.  The  remaining  data  base  available  at  DTIC  is  the 
Independent  Research  and  Development  (IR&D) . 

The  Technical  Report  data  bank  provides  bibliographic  descriptions 
of  technical  reports  prepared  under  DoD  funding.  The  data  bank  contains 


information  on  over  1.1  million  titles,  with  about  27,000  titles  added  in 
fiscal  year  1982.  As  shown  in  Table  1,  the  titles  added  in  1982  included 
about  3,000  that  were  classified,  8,000  that  were  unclassified/liniited,  and 
16,000  that  were  unclassifie<Vunlimited.  In  addition  to  services  from  the 
Technical  Reports  data  bank,  DTIC  also  provide  copies  of  the  actual  reports 
to  its  user  community. 

Research  and  development  activities  within  the  Ohited  States 
Government  and  their  associated  contractors,  subcontractors,  and  grantees, 
with  current  government  contracts,  are  eligible  to  receive  most  of  the 
infornation  from  DoD  data  bases  located  at  OTIC.  In  addition,  research  and 
development  organizations  without  current  contracts  may  become  eligible  for 
service  by  a  military  service  authorization  under  the  Defense  potential 
contractors  programs.  There  are  collections,  however,  which  contain 
proprietary  data  or  information  conpiled  for  the  specific  purpose  of  DoD 
management  decisions  which  are  made  available  to  Defense  components  only. 

The  information  collected  by  DTIC  is  provided  in  a  variety  of 
forms.  Technical  reports  are  distributed  both  cn  an  on-demand  basis,  in 
paper  form  and  microfiche,  and  automatically  through  the  Automatic  Document 
Distribution  (ADD)  program.  ADD  provides  microfiche  copies  of  newly  acces¬ 
sioned  reports  selected  according  to  a  user's  subject  interest.  As  indi¬ 
cated  in  Table  2,  DTIC  distributed  over  a  million  copies  of  technical 
reports  in  fiscal  year  1982,  376,000  in  response  to  on-demand  requests  and 
722,000  through  the  ADD  program.  Both  types  of  report  distribution  may  be 
to  libraries  within  the  DoD  organizations  or  directly  to  individuals. 

DoD  technical  reports  are  identified  by  users  through  a  number  of 
different  channels,  several  of  these  being  OTIC  products  and  services.  The 
most  frequently  used  method  of  identification  involves  searching  of  the 
online  Technical  Report  data  bank  through  the  Defense  RDT&E  online  system. 
Searches  may  either  be  performed  by  DTIC  or  on  remote  terminals  linked  to 
OTIC's  central  computer.  The  number  of  remote  sites  accessing  OTIC  is 
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TABLE  1 


Technical  Report  Data  Bank  Input  by  Level  of  Classification:  1982 


Reports  Input 


Level  of  Classification 

Nunber 

Percent 

Classified 

3,169 

12 

Uhclassified/limited 

8,048 

29 

Unclassified/unlimited 

16,089 

59 

Total 

27,306 

100 

Source:  Defense  Technical  Information  Center. 


TABLE  2 


Annual  Distribution  of  DITC  Products  and  Services:  1982 


DTIC  Product  or  Service 

Ft  1982 
Distribution 

On  Demand  Technical  Report  Distribution 

Microfiche 

Paper  Copy 

109,000} 
267. 0001 

Total 

376,000 

Automatic  Document  Distribution  (ADD) 

Microfiche 

722, 0001 

Online  Technical  Reports  Data  Bank  Interrogations 

DTIC 

Remote  Terminals 

214,000? 
409, 0001 

Total 

623, 0002 

Technical  Abstract  Bulletin  and  TAB  Indexes 

60.0003 

Current  Awareness  Bibliographies  (CAB) 

50.0001 

Management  Information  Data  Bank  Interrogations 

DTIC 

Remote  Terminals 

18,000} 

51 , 0001 

Total 

69.0002 

Sources : 


1  Defense  Technical  Information  Center  Surmary  Management  Data  Report. 

2  Estimated  by  King  Research,  Inc.  based  on  data  from  DITC's  Suraary 
Management  Data  Report. 

^  Defense  Technical  Information  Center  estimate. 


expanding  rapidly;  in  September  1981  there  were  422  terminals  in  the  net¬ 
work  and  in  September  1982  there  were  541  terminals.  As  indicated  in  Table 
2,  there  were  over  600,000  interrogations  of  the  Technical  Reports  data 
bank  in  1982,  with  over  400,000  of  these  frcm  remote  terminals. 

The  Technical  Report  data  bank  is  also  used  to  generate  subscrip¬ 
tion  bibliographic  products,  including  th  Technical  Abstract  Bulletin 
(TAB),  the  Technical  Abstract  Bulletin  Indexes  (TAB-I) ,  and  the  Current 
Awareness  Bibliography  (CAB)  program.  TAB  is  a  biweekly  listing  of  all  new 
classified  and  limited  scientific  and  technical  reports  received  by  DOTC 
within  the  processing  cycle.  It  is  available  as  a  bound  paper  document,  in 
microfiche,  or  on  nagnetic  tape.  As  noted,  TAB  covers  only  classified  and 
limited  reports;  unclassified/unlimited  reports  are  announced  through  the 
Ifational  Technical  Information  Services  (NTIS)  publication.  Government 
Reports  Announcements  and  Indexes  (GRA&I) . 

Issued  simultaneously  with  TAB  are  the  TAB  Indexes,  which  contains 
seven  indexes  to  the  documents  covered  by  TAB  and  also  to  unclassified/ 
unlimited  DoD  documents  announced  by  the  National  Technical  Information 
Service  (NTIS) .  TAB  Indexes  are  provided  in  paper  form.  The  1982  distri¬ 
bution  of  TAB  and  TAB  Indexes  was  about  60,000. 

The  CAB  program  provides  a  customized,  automated  bibliography 
service  based  on  the  recurring  subject  needs  of  OTIC  users.  Every  two 
weeks,  the  user's  subject  interest  profile  is  matched  against  information 
contained  in  newly  accessioned  documents  and  a  listing  of  citations  is 
provided  in  paper  form.  In  1982,  50,000  current  awareness  bibliographies 
were  distributed. 

OTIC's  management  information  data  banks  include,  the  Work  Unit 
Information  System  (WUIS) ,  the  R&D  Program  Planning  (R&DPP)  data  bank,  and 
the  Independent  Research  and  Development  (IR&D)  data  base.  WJIS  is  a 
collection  of  technically  oriented  summaries  describing  research  and  tech¬ 
nology  project  currently  in  progress  at  the  work  unit  level,  including 
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information  on  what,  where,  when,  by  whom,  at  what  costs  and  under  what 
sponsorship  research  is  being  performed.  R&DPP  is  a  repository  of  program 
planning  documentation  at  the  project  and  task  level.  The  IR&D  data  bank 
was  designed  to  enhance  camunication  between  DoD  scientists  and  engineers 
and  their  counterparts  in  industrial  organizations.  It  describes  technical 
programs  being  performed  by  DoD  contractors  as  part  of  their  independent 
research  and  development  programs. 

To  make  information  in  the  management  information  data  banks  avail¬ 
able  to  users,  OTIC  provides  several  categories  of  products  and  services. 
The  data  banks  are  available  online  through  the  ROT&E  online  system,  which 
again  may  be  searched  directly  from  the  user  site  or  from  OTIC.  Approx¬ 
imately  70,000  interrogations  of  the  three  management  information  data 
banks  were  made  in  1982,  including  51,000  from  remote  terminals  and  18,000 
by  DTIC.  Searches  by  OTIC  are  provided  on  an  on-demand  basis  and  in  the 
form  of  recurring  reports  compiled  periodically  according  to  a  user  profile 
and  in  a  user-defined  format. 

The  products  and  services  described  above  are  the  major  DTIC  pro¬ 
ducts,  and  were  specifically  addressed  in  our  study.  In  support  of  its 
mission,  OTIC  also  provides  other  products  and  services  —  among  them  the 
OTIC  Digest,  the  Bibliography  of  Bibliographies,  the  OTIC  Retrieval  and 
Indexing  Terminology  (DRIT) ,  referral  services,  a  central  registry,  train¬ 
ing  programs  for  online  system  users,  tours  and  briefings,  field  represen¬ 
tation,  and  review  of  OTIC  documents  —  not  directly  covered  in  this 
report. 
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CHAPTER  3 


ESTIMATED  USE  OF  OTIC  INFORMATION,  PRODUCTS  AND  SERVICES 


3.1  EEIC.  Users 

DTIC's  services  are  provided  primarily  to  research  and  development 
activities  within  the  United  States  Government  and  their  associated  con¬ 
tractors,  subcontractors,  and  grantees.  Such  groups  with  current  govern¬ 
ment  contracts  are  eligible  to  receive  most  of  the  information  from  DoD 
data  bases  located  at  DTIC,  and  research  and  development  organizations 
without  current  contracts  nay  become  eligible  for  service  by  a  military 
service  authorization  under  the  Defense  potential  contractors  program. 
Eligibility  for  services  is  determined  by  a  registration  process.  With 
registration,  organizations  are  provided  information  on  the  services 
offered  and  how  to  request  them.  Registration  for  OTIC  services  also 
assists  the  user  in  obtaining  the  services  offered  by  Defense-sponsored 
Information  Analysis  Centers  and  major  technical  libraries. 

DTIC  also  provides  some  limited  services  to  the  general  public  and 
to  foreign  requestors.  Unclassified  and  unlimited  DoD  documents  are 
announced  and  irade  generally  available  through  a  contractual  arrangenent 
with  NTIS.  For  foreign  requestors,  OTIC  provides  assistance  through 
correspondence  and  telephone  contacts  with  the  cognizant  foreign  embassies. 

In  this  study,  the  use  of  OTIC  services  by  the  primary  user  group 
—  DoD  scientists  and  engineers,  including  those  in  the  Federal  government 
and  those  in  contractor  organizations  —  was  studied.  An  early  effort  was 
addressed  to  identifying  the  number  of  organ izations  and  individuals  in 
this  group. 

About  three  thousand  organizations  are  currently  registered  with 
OTIC,  including  some  which  do  not  make  use  of  the  services  provided.  At 
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the  end  of  fiscal  year  1982,  there  were  190  subscribers  to  ADD,  335  sub¬ 
scribers  to  CAB,  and  541  terminals  in  user  organizations  able  to  access  the 
ROT&E  online  system.  The  largest  group  of  active  users  was  the  2,041 
organizations  provided  technical  reports  on  demand.  This  group  includes 
both  heavy  and  occasional  users  of  OTIC  products  and  services;  about  one 
third  of  the  group  ordered  fewer  than  ten  reports,  a  third  ordered  between 
ten  and  fifty,  and  the  final  third  ordered  more  the  fifty. 

The  number  of  individuals  within  user  organizations  was  estimated 
at  157,000.  This  is  based  on  dividing  the  estimated  $15.2  billion  expended 
on  DoD  research  activities  in  1982  by  an  average  $99,000  pet  researcher. 
Using  this  nunber,  the  average  number  of  individuals  per  user  organization 
can  be  estimated  at  78.  This  is  an  average;  the  organizations  vary  widely 
in  size.  In  general,  it  was  found  that  user  organizations  with  large 
numbers  of  on  demand  requests  also  had  large  numbers  of  individual  users. 
These  organizations  were  surveyed  more  heavily  in  this  study  so  that  each 
individual  user  would  have  an  approx iamtely  equal  chance  of  being  included 
in  the  sample. 

Of  the  DTIC  users  surveyed,  52  percent  were  in  Government  organiza¬ 
tions  and  48  percent  in  contractor  organizations.  Table  3  gives  the  pri¬ 
mary  work  role  of  those  surveyed.  As  indicated,  the  largest  group  was 
researchers,  with  one-quarter  engaged  primarily  in  management.  Small 
proportions  were  primarily  teachers  or  had  some  other  work  role  (such  as 
information  specialist) . 

3.2  Use  of  Journal  Articles.  Technical  Reports  and  Other  Materials 

It  is  estimated  that  there  are  at  least  157,000  scientists  and 
engineers  in  1982  engaged  in  research  and  development  funded  by  the  Depart¬ 
ment  of  Defense.  This  accounts  for  about  one-fourth  of  the  total  number  of 
680,000  scientists  and  engineers  engaged  in  R&D  in  the  United  States. 
Previous  studies  have  shown  that  scientists  and  engineers  (particularly 
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TABLE  3 


Primary  Work  Role  of  Government  and  DOD  Contractor 
Scientists  and  Engineers  Surveyed:  1982 


Primary 

Work  Role 

Percent  of 
DOD  Scientists 
and  Engineers 

Research 

59 

Management 

24 

Teaching 

7 

Other 

10 

Total 

100 

Source:  Defense  Technical  Information  Center  and  King  Research,  Inc.  Survey  of 
DTIC  Use,  1982. 


those  engaged  in  R&D)  spend  an  enormous  amount  of  time  reading.  Examples 
of  results  from  such  studies  are  given  in  Table  4. 

It  is  clear  that  scientists  and  engineers  spend  about  15  to  20 
percent  of  their  time  reading  the  literature;  about  5  percent  of  their  time 
is  devoted  to  reading  journal  articles.  Studies  by  King  Research  show  that 
a  significant  proportion  of  time  is  spent  reading  technical  reports  (9 
percent  in  energy,  12  percent  in  biomedicine  and  12  percent  by  DoD  scien¬ 
tists  and  engineers) .  This  report  is  largely  concerned  with  technical 
reports  prepared  and  read  by  Department  of  Defense  scientists  and 
engineers. 

One  of  the  objectives  of  this  study  was  to  determine  whether  scien¬ 
tists  and  engineers  funded  by  the  Department  of  Defense  depend  as  ex¬ 
tensively  on  the  literature  as  do  other  researchers.  In  particular,  the 
extent  to  which  they  use  the  information  products  and  services  provided  by 
the  Defense  Technical  Information  Service  (DTIC)  was  addressed. 

A  survey  of  scientists  and  engineers  funded  by  the  Department  of 
Defense  showed  that  these  persons  read  a  great  deal  and  that  they  devote  a 
substantial  amount  of  time  to  this  reading.  These  results  are  substanti¬ 
ated  by  other  results  observed  in  the  past.  The  estimated  average  annual 
number  of  readings  by  scientists  and  engineers  is  given  in  Table  5. 

The  amount  of  reading  by  scientists  and  engineers  funded  by  the 
Department  of  Defense  appears  to  be  about  the  same,  as  or  perhaps  slightly 
greater  than,  other  scientists  and  engineers.  In  total,  there  are  about 
12.4  million  readings  of  technical  reports  prepared  under  DoD  funding.  Of 
these  readings,  about  9.3  million  were  from  paper  copies  and  3.1  million 
from  microfiche  copies. 

Very  few  of  the  readings  of  technical  reports  are  what  one  might 
consider  to  be  light  reading  or  "just  to  get  the  idea."  In  fact,  respon¬ 
dents  reported  that  47  percent  of  their  last  readings  were  done  with  great 
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TABLE  4 


Time  Spent  by  Scientists  and  Engineers  Reading  Journals  and 
Other  Materials  by  Field  of  Science:  1960-1982 


Field  of  Science 

Hours  per  Month 
Reading  Journals 

Hours  per  Month 
Reading  All  Literature 

Physical  Sciences 

9.0 

25.1 

36.8 

11.7 

4.8-11.7 

24.3 

Engineers 

5.0 

19.1 

45.1 

2. 2-3.5 

8.6-13.8 

Life  Sciences  (cancer) 

8.1 

24.1 

Psychology 

4-8a 

13. 9a 

27.7 

Energy 

9.8 

23.6 

Biomedical 

7.6 

27.3 

Department  of  Defense 

10.5 

29.8 

Sources:  King  Research,  Inc.:  Scientific  Journals  in  the  U.S.,  Biomedical  Gom- 
munication  Expenditures,  and  Value  of  the  Energy  Data  Base;  Institute 
of  Technology,  An  Operations  Research  Study  of  the  Dissemination  and 
Use  of  Recorded  Scientific  Information  (Cleveland,  Ohio:  Case  Insti¬ 
tute  of  Technology,  1960);  B.  Weil,  "Benefits  frcm  Researcher  Use  of 
the  Published  Literature  at  the  Exxon  Research  Center,"  Paper  presented 
at  the  National  Information  Conference  and  Exposition,  Washington,  D.C. , 
April  20,  1977;  T.D.  Allen,  Managing  the  Flew  of  Scientific  and  Techno¬ 
logical  Information  (Cambridge,  Massachusetts:  Massachusetts  Institute 
of  Technology,  1966);  D.W.  King,  D.D.  McDonald,  and  C.H.  Olsen,  A  Survey 
of  Readers,  Subscribers  and  Authors  of  the  Journal  of  the  National  Can¬ 
cer  Institute  (Rockville,  Maryland:  King  Research,  1978);  American  Psy¬ 
chological  Association,  Reports  of:  The  American  Psychological  Associ¬ 
ation's  Project  on  Scientific  Information  Exchange  in  Psychology,  Vol.  1 
(Washington,  D.C.:  Anerical  Psychological  Association,  1963);  D.W,  King, 
and  N.K.  Roderer,  Systems  Analysis  of  Scientific  and  Technical  Communica¬ 
tion  in  the  United  States:  The  Electronic  Alternative  to  Connunication 
Through  Paper-Based  Journals  (Rockville,  Maryland:  King  Research,  1978) ; 
A.M.  Hall,  P.  Clague,  and  T.M.  Aitchison,  The  Effect  of  the  Use  of  an 
ADI  Service  in  the  Information-Gathering  Habits  of  Scientists  and  Tech¬ 
nologists  (London:  Institute  of  Electrical  Engineers,  1972),  p.  000. 

Computed  as  one-half  of  total  literature. 
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TABLE  5 


Average  Annual  Number  of  Readings1  per  Scientist  or  Engineer  of  Journal 
Articles,  Technical  Reports  and  Other  Materials  By  Department  of 
Defense,  Department  of  Energy  and  All  U.S.  Scientists  and  Engineers:  1982 


Scientists 
and  Engineers 
Funded  by 
Department  of 
Defense 
(1982) 

Scientists 
and  Engineers 
Funded  by 
Department  of 
Energy 
(1982)2 

All  Scientists 
and  Engineers 
(1977)3 

Journal  Articles 

152 

118 

105 

Technical  Reports 

Paper  Copy  (DOD) 
Microfiche  (DOD) 
Non-DOD 

116 

59 

20 

37 

110 

— 

Other  Materials 

23 

__ 

Respondents  were  told  that  reading  meant  going  beyond  the  table  of  con¬ 
tents,  title  and  abstracts  to  the  body  of  the  article  or  technical  re¬ 
port. 

The  Value  of  the  Energy  Database.  Donald  W.  King,  Jose-Marie  Griffiths,  Nancy 
K.  Roderer,  and  Robert  R.V.  Wiederkehr.  Submitted  to  the  Department  of  Energy 
Technical  Information  Center.  Rockville,  Maryland:  King  Research,  Inc. ,  March 
1982.  (NTS  No.  DE82-014250) . 

3 

Scientific  Journals  in  the  United  States:  Their  Production.  Use,  and  Economics, 
Donald  W.  King,  Dennis  D.  McDonald,  and  Nancy  K.  Roderer.  Stroudsburg,  PA: 
Hutchinson  Ross  Publishing  Company,  1981. 
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care  and  43  percent  of  the  last  readings  were  dene  in  enough  depth  to  pay 
attention  to  the  main  points  in  the  text.  The  scientists  and  engineers 
indicated  that  they  spent  an  average  of  about  1.7  hours  per  reading  of  the 
technical  reports  or  132  hours  on  the  average  reading  DoD  technical  reports 
over  a  year's  duration.  The  average  time  spent  reading  a  microfiche  copy 
was  about  one  hour  and  the  average  time  spent  reading  paper  copies  was 
closer  to  two  hours.  Hie  average  time  (per  reading)  spent  reading  other 
materials  included:  non-DoD  technical  reports,  1.7  hours;  journal 
articles,  50  minutes;  other  materials,  1.6  hours.  Hie  average  time  spent 
reading  by  Energy  scientists  and  engineers  was  1.5  hours  for  technical 
reports  and  one  hour  for  journal  articles  (which  was  about  the  average 
amount  of  time  per  reading  observed  for  all  scientists  and  enginers) . 
Hi  us,  results  of  the  DoD  survey  seem  to  be  consistent  with  data  observed 
elsewhere. 

Hie  scientists  and  engineers  were  asked  to  indicate  their  primary 
work  role  within  their  organization.  Most  of  them  were  engaged  in  research 
(59  percent)  with  the  reminder  involved  in  management  (24  percent) ,  teach¬ 
ing  (7  percent)  or  sane  other  role  (10  percent) .  As  might  be  expected,  the 
amount  of  reading  of  DoD  technical  reports  varies  substantially  by  these 
work  roles.  Those  engaged  primarily  in  teaching  read  the  most.  They 
averaged  102  readings  per  year  (most  of  which  are  from  paper  copies)  com¬ 
pared  to  slightly  over  70  readings  per  year  by  those  involved  in  research 
and  management. 

In  Table  6,  the  estimated  readings  per  DoD-funded  scientist  or 
engineer  are  multiplied  by  the  number  of  scientists  and  engineers  (157,000) 
to  obtain  the  total  amount  of  reading  by  the  group.  As  shown,  there  are 
over  45  million  readings  by  the  group,  with  52  percent  of  these  being  of 
journal  articles,  40  percent  technical  reports,  and  8  percent  other  materi¬ 
als.  Over  cne-quarter  of  all  readings  are  of  DoD  sponsored  technical 
reports. 
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TABLE  6 


Total  Readings  of  Journal  Articles,  Technical  Reports, 
and  Other  Materials  by  DOD- Funded  Scientists  and  Engineers:  1982 


Type  of  Material 


Total  Readings 
(millions) 


Journal  Articles 

23.9 

Technical  Reports 

18.2 

DOD  (paper  copy) 

9.3 

DOD  (microfiche) 

3.1 

Non-DOD 

5.8 

Other  Materials 

3.6 

Source:  Defense  Technical  Information  Cent<  r  and  King  Research,  Inc.  Survey 
of  DTIC  Use,  1982. 


Another  indicator  of  information  use  is  the  time  spent  reading. 
Tbble  7  gives  the  amount  of  time  spent  per  item  and  per  user  and  the  total 
time  spent  annually  by  all  users.  As  would  be  expected,  the  average  time 
spent  reading  a  technical  report  (1 .7  hours,  or  about  100  minutes)  is 
larger  than  the  50  minutes  spent  reading  a  journal  article.  Distinguishing 
between  paper  and  microfiche  copies  of  technical  reports,  more  time  on  the 
average  is  spent  reading  a  paper  copy  report.  The  total  time  spent  by  an 
individual  user  annually  is  558  hours,  about  17  percent  of  all  work  time. 
Of  this  total,  about  35  percent  is  spent  on  journal  articles,  55  percent  on 
technical  reports,  and  10  percent  on  other  materials.  About  37  percent  of 
the  DoD  users  time  spent  on  reading  is  associated  with  DoD  technical 
reports. 


Tbble  7  also  gives  the  total  time  spent  reading  by  DoD  contractors. 
For  all  types  of  materials,  this  is  56  million  hours  or  about  27,000  person 
years  annually. 

It  seems  clear  that  scientists  and  engineers  funded  by  the  Depart¬ 
ment  of  Defense  depend  a  great  deal  on  technical  reports  and  other  publica¬ 
tions  for  their  research  as  well  as  for  their  management  activities  and 
their  continued  education.  In  the  next  section  we  describe  how  these 
persons  identify  and  gain  access  to  technical  reports  and  the  extent  of  the 
role  played  by  the  Defense  Technical  Information  Center  in  these  informa¬ 
tion  transfer  functions.  Then,  in  the  following  section,  we  indicate  the 
value  of  information  found  in  DoD  technical  reports  and  the  value  added  by 
DTIC  information  products  and  services. 

3.3  Ose  of  OTIC  Products  and  Services 

The  DTIC  products  and  services  covered  specifically  in  the  survey 
of  users  included  technical  reports,  online  searches,  TAB,  CAB,  and  the 
management  data  banks.  Online  searches,  TAB  and  CAB  are  each  used  to 
identify  DoD  technical  reports  which  then  provide  the  user  with  primary 
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TABLE  7 


Time  Spent  by  DOD- Funded  Scientists  and  Engineers  Reading  Journal 
Articles,  Technical  Reports,  and  Other  Materials:  1982 


Reading  Time 

(Hours) 

Type  of  Material 

Per  Item 

Per  User 

Annual 

Total 

(millions) 

Journal  Articles 

.8 

127 

20.0 

Technical  Reports 

DOD  (paper  copy) 

1.9 

111 

17.4 

DOD  (microfiche) 

1.1 

21 

3.3 

Non-DOD 

1.7 

62 

9.7 

Other  Materials 

1.6 

37 

5.8 

Total 

1.2 

358 

56.2 

Source:  Defense  Technical  Information  Center  and  King  Research,  Inc.  Survey 
of  OTIC  Use,  1982. 


information  on  conpleted  research.  The  managenent  data  banks  also  provide 
primary  information  covering  planned  and  ongoing  research.  The  use  made  of 
each  of  these  product  categories  is  covered  in  turn  below. 

As  indicated  earlier,  DoD  users  read  a  total  of  12.4  million  DoD 
technical  reports  annually,  or  an  average  of  79  per  individual.  These 
readings  are  looked  at  for  users  in  different  organizations  and  work  roles 
in  Tables  8  and  9.  From  Table  8,  the  average  number  of  readings  by 
Government  users  is  63,  while  readings  by  contractors  average  96.  The 
level  of  reading  also  varies  by  work  role  with  those  users  identifying 
teaching  as  their  primary  work  role,  reading  about  50  more  reports,  on  the 
average,  than  researchers  or  managers.  The  average  number  of  reports  read 
by  researchers  and  managers  was  about  the  same. 

In  order  to  read  a  technical  report,  a  user  must  first  find  out 
about,  that  is,  identify  it,  and  then  obtain  a  physical  copy.  In  some 
cases  these  two  activities  of  identification  and  access  occur  simultane¬ 
ously,  as  for  example  when  a  report  is  received  through  some  sort  of  stan¬ 
dard  distribution  and  identified  by  the  user  as  being  of  interest.  For  the 
most  part,  however,  identification  and  access  occur  separately.  Tables  10 
and  11  give  breakdowns  of  the  total  DoD  report  readings  by  identification 
and  access  channels. 

In  Table  10,  the  identification  methods  shown  include  online 
bibliographic  searches,  printed  indexes,  CAB,  citations  in  reports  or 
journals,  colleague  referrals,  reports  routed  to  users,  and  other.  The 
most  frequently  used  of  these  is  the  online  bibliographic  search,  which 
accounts  for  nearly  half  of  all  readings.  For  the  most  part  these  are 
searches  of  the  Technical  Report  data  bank  using  the  RDT&E  system,  with 
searches  performed  either  by  the  user's  organization  or  by  DTIC. 

Users  may  also  identify  technical  reports  through  TAB  and  CAB,  two 
other  DTIC  products.  In  combination  with  online  searches,  use  of  these 
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TABLE  3 


Annual  Nmber  of  Readings  of  DoD  Technical  Reports  by 
Goverrment  and  DOD  Contractor  Scientists  and  Engineers:  1982 


Annual  Technical 

Report  Readings 

User  Category 

Average 

Total 

(millions) 

Government 

63 

5.2 

DOD  Contractor 

96 

.7.2 

Total 

79 

12.4 

TABLE  9 

Annual  Number  of  Readings  of  DOD  Technical  Reports  by 
DOD-?unded  Scientists  and  Engineers  in  Four  Work  Roles:  19S: 

Annual  Technical  Report  Readings 

User  Work  Role 

Average 

Total 

(millions) 

Research 

68 

6.3 

Management 

72 

2.7 

Teaching 

105 

1.2 

Other 

142 

2.2 

Total 

79 

12.4 

Source:  Defense  Technical  Information  Center  and  King  Research,  Inc.  Survey 
of  DTIC  Use,  1982. 
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TABLE  10 


Annual  Nmber  of  Readings  of  DOD  Technical  Reports  by  Government  and 
DOD  Contractor  Scientists  and  Engineers  by  Identification  Method:  1982 


Annual  Technical 
_ Report:  Readings _ 

Nmber  Percent  of 

_ Identification  Method _  (millions)  Total 

Bibliographic  Search 

Done  by  Organization  5.1  41 

Done  by  DITC  .9  7 

Printed  Index 

TAB  1.5  12 

Other  1.0  8 

CAB  (Current  Awareness  Bibliography)  .7  6 

Citation  in  a  Report  or  Journal  1.1  9 

Colleague  Referral  1.0  8 

Routed  to  User  .5  4 

Other  ._6  5 

Total  12.4  100 

Source:  Defense  Technical  Information  Center  and  King  Research,  Inc.  Survey 
of  OTIC  Use,  1982. 
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TABLE  11 


Annual  Nuiber  of  Readings  of  DOD  Technical  Reports  by  Government  and 
DCD  Contractor  Scientists  and  Engineers,  by  Distribution  Method:  1982 


Distribution  Method 


Annual  Technical 
Report  Readings _ 

Nurber  Percent  of 

(millions)  Total 


Standard  Distribution  to  Individual 

.5 

4 

Standard  Distribution  to  Library 

.2 

2 

Ordered  by  Individual 

.6 

5 

Ordered  by  Library 

10.2 

82 

Other 

.9 

_ 7 

Total 

12.4 

100 

Source:  Defense  Technical  Information  Center  and  King  Research,  Inc.  Survey  of 
DTIC  Use,  1982. 
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identification  channels  yield  a  total  of  66  percent  of  all  readings 
identified  through  DTIC  products  or  services.  Identification  through  other 
printed  indexes/  which  accounts  for  8  percent  of  all  readings/  involves  the 
use  of  NASA's  STAR,  NTIS's  GRA& I ,  and  other  publications.  In  total,  about 
three-quarters  of  report  identifications  involve  a  bibliographic  product  or 
service,  with  the  remaining  quarter  made  up  of  identification  via  citation 
(9  percent)  colleague  referral  (8  percent),  routing  (4  percent)  and  other 
means  (5  percent) . 

Table  11  breaks  down  the  total  report  readings  by  distribution 
channel.  There  are  many  ways  (or  channels)  through  which  copies  of  the 
technical  reports  are  distributed  including: 

(1)  Directly  to  individuals  through  standard  distribution. 

(2)  To  individuals  through  libraries  that  obtained  copies 
through  standard  distribution. 

(3)  Directly  to  individuals  through  their  ordering  it  from 
DTIC. 

(4)  To  individuals  through  libraries  that  obtained  copies  by 
ordering  them  for  the  individual  or  someone  previously. 

(5)  Through  other  means  such  as  a  colleague,  the  National 
Technical  Information  Service,  etc. 

By  far  the  greatest  proportion  of  readings,  over  80  percent,  is  of 
reports  ordered  by  libraries.  In  total,  87  percent  of  readings  are  of 
reports  ordered  on  demand,  6  percent  are  of  reports  received  through  stan¬ 
dard  distribution,  and  7  percent  are  accessed  through  other  means. 

Table  12  compares  the  number  of  readings  from  standard  and  on 
demand  distribution  with  the  number  of  copies  of  reports  distributed  via 
these  means.  Data  on  both  distribution  and  readings  apply  to  1982. 
Strictly  speaking,  they  cannot  be  combined  to  yield  the  average  readings 
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TABLE  12 


Estimated  Readings  per  DOD  Technical  Report  Copy  Distributed 
by  Distribution  Method:  1982 


Distribution  Method 

Ninber  of 
Copies 
Distributed 
(thousands) 

Nunber  of 
Readings 
(millions) 

Average 
Readings 
Per  Copy 

Standard  Distribution 

722 

.7 

1 

On  Demand  Distribution 

376 

» 

10.7 

28 

All  Methods 

1,098  .  ' 

12.4 

11 

Source:  Defense  Technical  Information  Center  and  King  Research,  Inc.  Survey 
of  uric  Use,  1982. 
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per  copy  because  not  all  readings  of  reports  distributed  in  1982  will  occur 
in  1982  and  not  all  readings  occurring  in  1982  will  be  of  1982  reports. 
Since  the  levels  of  distribution  and  reading  are  fairly  constant  over  the 
years  however,  we  can  combine  the  data  to  approximate  readings  per  copy. 
Hie  result  shows  about  one  reading  per  copy  for  standard  distribution  (all 
microfiche)  and  28  readings  per  copy  for  on  demand  distribution  (30  percent 
microfiche  and  70  percent  paper  copy) .  In  total,  there  are  about  11  read¬ 
ings  of  DoD  technical  reports  per  copy  distributed.  This  can  be  compared 
with  an  estimated  1.1  readings  per  technical  report  copy  distributed  by  the 
Department  of  Biergy  and  .1  readings  per  article  copy  distributed  for 
scientific  and  technical  journal  articles  generally. 

Report  readings  can  also  be  categorized  according  to  the  level  of 
classification  of  the  report,  that  is,  classified,  unclassified/limited,  or 
unclassified/unlimited.  This  is  done  in  Table  13.  Almost  60  percent  of 
report  readings  are  of  unclassified  and  unlimited  materials,  with  about  20 
percent  each  for  classified  and  limited  reports  respectively.  Hie  20 
percent  of  readings  being  of  classified  reports  can  be  compared  to  a  sim¬ 
ilar  percent  for  on  demand  distribution  of  classified  reports  (most  read¬ 
ings  come  through  on  demand  distribution) ,  suggesting  that  the  number  of 
readins  per  copy  distributed  is  similar  for  classified  and  unclassified 
reports . 


Why  do  DoD  scientists  and  engineers  read  technical  reports?  Gen¬ 
eral  categories  of  purposes  are  to  support  education,  research,  and  manage¬ 
ment.  In  the  survey  of  DoD  scientists  and  engineers,  users  were  given  the 
opportunity  to  indicate  one  or  more  purposes  for  their  last  reading  of  a 
DoD  report.  On  the  average,  users  mentioned  about  four  purposes.  As  shown 
in  Table  14,  the  most  frequently  acknowledged  purposes  were  application  of 
a  report  findings  to  a  current  project  (75  percent)  and  professional 
development,  current  awareness  or  general  interest  (74  percent) .  About 
half  mentioned  application  of  the  report's  methodology  to  a  current 
project,  use  for  citation,  and  use  in  preparing  some  publication.  Conparing 


TABLE  13 


Annual  Number  of  Readings  of  Classified,  Unclass if ied/Limi ted  and 
Unclassified/Unlimited  Technical  Reports  by  DOD- Funded 
Scientists  and  Engineers:  1982 


Annual  Technical 
Report  Readings 


Level  of  Report 
Classification 

Nunber 

(millions) 

Percent  of 
Total 

Classified 

2.6 

21 

Unclass  if  ied/Limited 

2.6 

21 

Unclassified/Unlimited 

7.2 

58 

Total 

12.4 

100 

Source:  Defense  Technical  Information  Center  and  King  Research,  Inc.  Survey 
of  DTIC  Use,  1982. 


TABLE  14 


Proportion  of  Scientists  and  Engineers  Rinded  by  the  Department  of  Defense  and 
Department  of  Energy  Who  Indicated  Various  Purposes  of 
Reading  DOD  Last  Read  Technical  Reports:  1982 


Purposes  of  Reading  Last  Read  Technical  Reports 
Educational 


Proportion  of  Yes  Responses 

O) 


Department 

of 

Defense 


Department 

of 

Energy 


Self  —  For  professional  development,  cur¬ 
rent  awareness  or  general  interest  74  75 

Others  —  In  preparation  of  a  lecture  or 

presentation  39  40 


Research 


In  preparation  of  a  research  proposal  27  38 

To  apply  its  findings  to  a  current  project  75  77 

To  apply  its  methodology  to  a  current  project  53  50 

In  preparation  of  an  article,  book,  review  or 

report  49  50 

As  a  citation  in  an  article,  book,  review  or 

report  50  48 


Management 

For  the  planning,  budgeting  and  management 
of  research 
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40 


Other 


6 


Source:  Defense  Technical  Information  Center  and  King  Research,  Inc.  Survey  of 
DTIC  Use,  1982. 
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the  purposes  of  use  of  DoD  reports  with  those  of  DOE  reports,  also  shown  in 
Table  14,  suggests  very  similar  patterns  of  use  for  the  two  groups. 

Table  15  further  describes  DoD  report  readings  by  indicating  the 
depth  to  which  each  report  was  read.  As  suggested  by  the  average  reading 
time  of  100  minutes,  most  reports  were  read  either  with  great  care  (47 
percent)  or  with  attention  to  the  main  points  (43  percent) .  Ten  percent  of 
readings  were  nade  just  to  get  the  idea  of  the  report. 

To  look  at  DoD  scientists'  and  engineers'  utilization  of  DTIC 
bibliographic  products  and  services  in  more  depth,  separate  survey  sections 
addressed  online  bibliographic  searching,  use  of  TAB,  and  use  of  CAB. 

As  indicated  earlier,  online  searches  may  be  performed  by  DTIC  or 
by  the  user  organizations,  increasingly  in  recent  years  by  the  latter.  In 
1982,  as  shown  in  Table  16,  about  two-thirds  of  the  interrogations  made  of 
the  Technical  Report  data  bank  were  made  by  user  organizations.  In  total, 
over  600,000  interrogations  were  made.  DTIC  estimates  that  there  are,  on 
the  average,  about  three  interrogations  per  search. 

When  a  search  is  performed,  multiple  citations  to  reports  of  poten¬ 
tial  relevance  are  obtained.  The  average  search  of  the  TR  data  bank, 
according  to  users,  yields  about  170  citations.  This  is  narrowed  down  by 
the  user  to  a  number  of  titles  deemed  relevant.  For  the  average  DoD  user, 
the  number  of  relevant  titles  per  search  is  estimated  at  39,  or  about 
one-quarter  of  the  titles  output  from  the  online  search. 

Not  all  reports  identified  as  relevant  are  actually  obtained  by 
users,  and  not  all  reports  obtained  are  read.  For  the  average  search,  with 
39  reports  identified,  15  reports  are  accessed  and  10  read  (see  Table  17) . 
It  is  the  case  with  DoD  users,  as  with  scientific  and  technical  users 
generally,  that  the  number  of  readings  resulting  from  an  online  search  is 
greater  than  that  resulting  from  a  printed  index  search. 
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TABLE  15 


Nmber  of  Readings  of  DOD  Technical  Reports  by  Reading  Depth:  1982 


Reading  Depth 

Number  of 
Readings 
(millions) 

Percent  of 
Readings 

With  Great  Care 

5.8 

47 

With  Attention  to  Main  Points 

5.3 

43 

Just  to  Get  Idea 

1.3 

10 

Total 

12.4 

100 

Source:  Defense  Technical  Information  Center  and  King  Research,  Inc.  Survey  of 
IEIC  Use,  1982. 
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TABLE  16 


Nixher  of  Online  Interrogations  of  the  Technical  Reports  Data  Bank  By  DTIC 

and  by  Other  Organizations:  1932 


Searching  Organization 
DTIC 


Remote  Site 

Total 


Interrogations  of  the 
Technical  Report  Data  Bank 

214,000 

409,000 

623,000 


Source:  King  Research ,  Inc.  Estimate  based  on  DTIC's  Sunniary  Management  Data 
Report 


37 


TABLE  17 


I 


Source: 


Nunber  of  DOD  Technical  Reports  Identified,  Accessed  and 
Read  as  a  Result  of  an  Online  Search:  1982 


Nunber 

Percent 
of  Reports 

per  Search 

Identified 

Reports  Identified 

38.7 

— 

Reports  Accessed 

14.7 

38 

Reports  Read 

11.8 

30 

Defense  Technical  Information  Center  and  King  Research,  Inc.  Survey 
of  DnC  Use,  1982. 
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Online  searches  may  be  performed  directly  by  the  user  or  by  an 
intermediary  such  as  a  librarian.  In  the  case  of  searches  of  the  TR  data 
base,  95  percent  of  all  searches  were  performed  by  someone  other  than  the 
end  user.  Thus,  when  users  were  asked  about  the  amount  of  time  spent 
searching,  the  bulk  of  the  time  reported  was  associated  with  making  the 
search  request  and  with  reviewing  search  output.  As  shown  in  Table  18,  a 
total  of  2.4  million  hours  were  spent  on  online  searching.  This  is  about 
15  hours  per  user  or  almost  four  hours  per  search.  Considering  only  the 
items  read  as  a  result  of  the  search,  about  25  minutes  were  spent  by  the 
user  for  each  item  that  wa ;  ultimately  read. 

Another  means  of  identifying  DoD  reports  is  through  a  TAB  search. 
In  1982,  there  were  an  estimated  1.5  million  uses  of  TAB,  or  an  average  of 
9.5  per  user.  This  is  25  readings  for  every  biweekly  copy  of  TAB 
distributed. 

TAB  and  TAB  Indexes  are  used  by  a  analler  proportion  of  the  user 
population  than  are  online  searches,  about  30  percent  of  the  population  as 
compared  with  65  percent.  Among  users  of  TAB,  more  than  half  had  used  the 
publication  within  the  last  two  weeks  (probably  the  last  biweekly  issue) . 
About  half  were  browsing  when  they  used  TAB,  and  about  half  conducted  a 
problem-related  search.  These  data  seem  to  suggest  that  TAB  uses  are  about 
equally  divided  between  currant  awareness  purposes  and  retrospective  sub¬ 
ject  searches. 

The  average  number  of  items  identified  as  relevant  through  a  TAB 
search  is  about  14.  As  shown  in  Table  19,  about  six  of  these  titles  are 
subsequently  accessed  and  about  two  are  actually  read.  The  low  percentage 
of  reports  read  is  probably  because  so  many  of  the  searches  were  very 
recent;  users  were  asked  to  indicate  completed  rather  than  planned 
readings. 

The  average  time  spent  can  a  TAB  search  was  reported  by  users  as 
about  45  minutes.  This  is  about  20  minutes  per  item  read,  as  compared  with 
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TABLE  18 


Source:  Defense  Technical  Information  Center  and  King  Research,  Inc.  Survey 
of  DTIC  Use,  1982. 
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25  minutes  for  online  searching.  Total  time  spent  on  TAB  searching  by  all 
users  was  1.1  million  hours 

DTIC's  CAB  program  provides  a  customized,  automated  bibliography 
service  based  on  the  recurring  subject  needs  of  its  users.  The  service  is 
bi-weekly.  At  the  end  of  1982,  there  were  335  CAB  subscribers  including 
both  individuals  and  institutions.  About  9  percent  of  DoD  users,  or  an 
estimated  14,000  individuals,  report  having  used  CAB  within  the  last  year. 
Each  of  these  reported  22  uses  on  the  average,  for  an  annual  total  of 
314,000  uses.  These  uses  resulted  in  about  700,000  readings. 

As  implied  by  its  title,  CAB  is  used  mostly  for  current  awareness. 
With  26  bi-weekly  copies  distributed  per  year  and  22  uses  reported,  it 
appears  that  most  users  are  regular  users.  The  results  of  an  average  CAB 
use  are  given  in  Table  20:  about  five  titles  identified  as  relevant,  three 
titles  accessed,  and  just  slightly  fewer  titles  read.  In  part,  the  high 
proportion  of  reports  identified  that  are  actually  read  is  probably  due  to 
the  broad,  current  awareness  focus  of  CAB. 

For  online,  TAB  and  CAB  searches,  DoD  users  were  asked  to  indicate 
a  breakdown  of  reports  by  classification  level.  This  was  done  for  both 
reports  identified  as  relevant  and  for  reports  read.  The  purpose  was  to 
look  for  different  identification  patterns  for  different  classification 
levels . 

As  shown  in  Table  21,  somewhat  different  breakdowns  by  classifica¬ 
tion  level  were  observed  for  reports  identified  and  reports  read.  More 
significant,  however,  were  the  differences  between  online  searching,  TAB 
and  CAB  use.  Nearly  half  of  the  reports  read  as  a  result  of  a  CAB  iden¬ 
tification  were  limited,  which  was  true  of  only  20  percent  of  reports 
identified  through  TAB  and  7  percent  of  those  identified  by  an  online 
search.  Reports  identified  through  an  online  search  were  more  likely  to  be 
unclassified  and  unlimited  than  with  either  of  the  other  two  bibliographic 
products . 


TABLE  20 


Nunber  of  DOD  Technical  Reports  Identified,  Accessed  and  Read  as  a 

Result  of  a  CAB  Use:  1982 


Nunber 

Percent 
of  Reports 

per  Search 

Identified 

Reports  Identified 

A. 5 

— 

Reports  Accessed 

3.0 

67 

Reports  Read 

2.7 

60 

Source:  Defense  Technical  Information  Center  and  King  Research,  Inc.  Survey 
of  DTIC  Use,  1982. 


TABLE  21 


Comparison  of  Classification  Level  of  DOD  Technical  Reports  Identified  and 
Read  as  a  Result  of  Online,  TAB,  and  CAB  Searches :  1982 


Percent  of 
Reports 

Identification  Msthod  and  Classification  Level  Identified 


Percent  of 
Reports 
Read 


Online  Search 


TAB 


CAB 


Classified  Reports 

15 

30 

Unclassified/Limited  Reports 

20 

7 

Unclassified/Unlimited  Reports 

66 

63 

and  TAB  Indexes 

Classified 

51 

30 

Unclassified /Limited  Reports 

14 

20 

Unclassified/Unlimited  Reports 

35 

50 

Classified 

18 

15 

Uhclassified/Limited  Reports 

28 

44 

Unclass  if  ied/ Unlimited  Reports 

54 

41 

Source:  Defense  Technical  Information  Center  and  King  Research,  Inc.  Survey  of 
Dnc  Use,  1982. 
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Table  22  compares  the  readings  identified  through  OTIC  biblio¬ 
graphic  products  with  those  from  other  sources.  Looking  at  the  total 
report  readings  fran  online  searching,  TAB  or  CAB,  we  find  29  percent 
classified,  12  percent  unclassified/limited,  and  59  percent  unclassified!/ 
unlimited.  For  other  identification  methods,  the  proportions  are  5  percent 
classified,  38  percent  unclassified/limited,  and  57  percent  unclassified/ 
unlimited.  The  bulk  of  readings  of  classified  reports  cane  from  identifi¬ 
cations  via  a  OTIC  bibliographic  product,  while  limited  reports  are  more 
likely  to  be  identified  by  some  other  means.  Both  categories  of  identifi¬ 
cation  methods  are  equally  likely  to  result  in  reading  of  an  unclassified 
and  unlimited  report. 

In  addition  to  services  associated  with  its  Technical  Reports  data 
bank,  OTIC  also  maintains  and  provides  services  from  its  three  management 
data  banks:  WUIS,  R&DPP,  and  IR&D.  One  hundred  thirty-one  users  responded 
to  the  management  databank  survey  conducted  as  a  part  of  the  project.  WUIS 
users  totaled  72  (or  55  percent  of  databank  respondents) ,  R&DPP  users 
totaled  24  (18  percent)  and  IR&D  totaled  35  (27  percent) . 

Table  23  shews  the  number  of  uses  of  OTIC  management  database  by 
scientists  and  engineers.  For  the  R&D  Work  Unit  Information  System  (WUIS) 
the  respondents'  uses  totaled  207,000  with  an  average  of  1.3  uses  per  year. 
For  the  R&D  Program  Planning  Data  Base  (R&DPP)  there  were  50,000  uses  per 
year  with  an  average  of  .3  uses  per  respondent  per  year.  The  Independent 
Research  and  Development  Data  Base  (IR&D)  respondents  reported  23  thousand 
uses  per  year  with  an  average  of  .2  uses. 

The  survey  respondents  were  also  asked:  "Did  you  pass  the 
information  you  obtaind  on  to  anyone  else?"  The  average  number  of  pass-ons 
(Table  24)  per  WJIS  user  was  4.1.  The  R&DPP  database  users  reported  an 
average  of  2.3  pass-ons  while  IR&D  users  averaged  3.1  pass-ons  of 
information  froi  a  particular  search. 
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TABLE  22 


DQD  Technical  Report  Readings  by  Identification  Method  and 
Report  Classification  Level:  1982 


_ Classification  Level _ 

Unclassified/  Unclassified/ 
Classified  Limited  Unlimited 


All  Readings  (Millions) 

2.6 

2.6 

7.2 

Proportion  of  Readings  (%) 

21 

21 

58 

Readings  Identified  through  DTIC 

Bibliographic  Products1  (Millions) 

2.4 

1.0 

4.8 

Proportion  of  Readings  (7«>) 

29 

12 

59 

Readings  Identified  through 

Other  Means-^  (Millions) 

.2 

1.6 

2.4 

Proportion  of  Readings  (%) 

5 

38 

57 

Includes  online  searching  of  the  TR  data  bank,  TAB  use,  and  CAB  use. 

2 

Includes  use  of  other  bibliographic  products,  citation,  colleague  referral, 
routine,  and  other. 

Source:  Defense  Technical  Information  Center  and  King  Research,  Inc.  Survey  of 
DTIC  Use,  1982. 
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Table  23 


Annual  Number  of  Uses  of  OTIC  Management  Data  Banks 
by  DOD  Scientists  and  Engineers:  1982 


Annual  Uses 


Data  Bank 

Average 

Der  year 

W 

Total 

fthr»v<5andc;^ 

R&T  Work  Unit 

1.3 

207 

R&D  Program  Planning 

.3 

50 

Independent  R&D 

_^2 

_21 

TOTAL 

1.8 

280 

Source:  Defense  Technical  Information  Center  and  King  Research,  Lie.  Survey  of 
DTIC  Use,  1982. 


TABLE  24 


Nunber  of  Pass-on  Uses  of  OTIC  Management  Data  Banks:  1982 


Average 

Data  Bank 

Pass -On  Uses 

ROT  Work  Unit 

4.1 

R&D  Program  Planning 

2.3 

Independent  R&D 

3.1 

Total 

3.7 

Source:  £=£®1fTe  Te?££cal  Information  Center  and  King  Research,  Lie.  Survey  of 
OTIC  Use,  1982.  J 
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The  usefulness  of  DTIC  management  data  banks  was  investigated 
(Table  25) .  Ninety-four  percent  of  the  users  queried  reported  that  the 
results  of  the  search  were  useful  while  only  6  percent  felt  that  the  data 
was  not  useful.  Of  the  R&DPP  users,  83  percent  found  the  results  useful 
and  17  percent  did  not.  A  majority  of  the  IR&D  users  also  said  the  results 
were  useful  (90  percent)  with  only  10  percent  reporting  no  use.  The 
reasons  for  usefulness  as  reported  by  users  are  listed  in  Appendix  A. 

Completeness  of  the  information  found  during  the  user's  searches 
was  also  measured  (Table  26) .  Seventy-one  percent  of  WUIS  users  said  "yes" 
whie  14  percent  said  "nor  and  15  percent  did  not  know.  Sixty-two  percent 
of  R&DPP  users  said  "yes,"  17  percent  said  "no"  and  21  percent  did  not 
know.  Of  the  IR&D  users,  72  percent  found  the  information  complete  while  11 
percent  did  not  and  17  percent  responded  "don't  know." 

Overall,  nine  percent  of  the  DoD  researchers  and  engineers  surveyed 
used  DTIC's  management  data  bases.  This  is  about  14,000  users,  not 
including  those  to  whom  the  information  is  passed  on.  Since  a  total  of 
280,000  uses  are  estimated,  the  average  number  of  uses  per  user  is  20. 
Mien  asked  about  future  use  of  the  management  data  banks,  current  users 
anticipated  modest  increases  in  usage  over  time. 


TABLE  25 


Source: 


Rating  of  Usefulness  of  DITC  Managanent  Data  Banks:  1982 


Were  Results  Useful 
(Percent) _ 


Data  Bank 

Yes 

No 

R&T  Work  Unit 

94 

6 

R&D  Program  Planning 

83 

17 

Independent  R&D 

90 

10 

Total 

92 

8 

Defense  Technical  Information  Center  and  King  Research,  Inc.  Survey  of 
Erne  Use,  1982. 


49 


CHAPTER  4 


ESTIMATED  VALUE  OF  DOD  INFORMATION  MD  OTIC  INFORMATION 
PRODUCTS  AND  SERVICES 


4.1  felue  In  Terms  of  Readers'  Willingness  to  Pay  for  Information 

In  Section  3,  it  is  found  that  DoD  scientists  and  engineers  devote 
a  substantial  amount  of  their  time  acquiring,  reading  and  using  information 
found  in  DoD  technical  reports.  When  this  time  and  effort  is  expressed  in 
terms  of  expenditures,  it  is  found  that  they  are  willing  to  pay  a  great 
deal  for  the  information.  The  estimated  effective  price  that  they  actually 
pay  is  found  in  Table  26. 

Thus,  the  total  value  that  DoD  users  are  willing  to  pay  for  infor¬ 
mation  generated  by  DoD  is  estimated  to  be  about  $560  million  or  an  average 
of  about  $3,600  per  scientist  and  engineer.  The  amount  of  this  value 
directly  attributable  to  the  use  of  OTIC  products  or  services  (i.e.,  TAB, 
CAB  or  computer  searching)  is  estimated  to  be  about  $370  million  or  an 
average  of  $2,300  per  scientist  and  engineer. 

These  values  do  not  take  into  account  that  DoD  scientists  and 
engineers  could  identify  and  obtain  their  information  in  ways  other  than 
IAB,  CAB  or  OTIC  bibliographic  searching.  To  obtain  the  effective  price 
with  alternate  means  of  searching,  costs  for  the  specific  identification 
method  (from  Table  26)  are  added  to  the  general  purchase,  processing, 
access  and  reading  cost.  For  example,  the  cost  of  manually  searching  by 
other  means  is  estimated  to  be  $75.40  rather  than  $49.90  for  TAB  and  $44.90 
for  CAB,  and  substitition  for  OTIC  computer  bibliographic  searching  is 
estimated  by  $51.10  instead  of  $43.50.  (See  reference  [1]  for  details  on 
how  these  substitutes  costs  are  estimated) .  Value  of  each  of  the  products 
and  services  is  estimated  below  by  calculating  the  cost  of  changes  in 
searching  practices  resulting  from  withdrawal  of  a  product  or  service.  It 
is  assumed  that  a  given  amount  of  expenditures  will  be  made  by  a  user  (or 
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TABLE  26 


Effective  Price  of  Reading  DoD  Technical  Reports  by 
DoD -Funded  Scientists  and  Engineers:  1982 


Cost  Category 

Average 

Per 

Reading 

($) 

Number  of 
Readings 
(Millions) 

Total 
Value 
to  Readers 
($  Millions) 

Report  Purchase 

$  0.10 

12.4 

$  1.2 

Processing 

3.30 

12.4 

40.9 

Identification 

TAB 

12.30 

1.5 

18.5 

CAB 

7.30 

0.7 

5.1 

Search 

5.90 

6.0 

35.4 

Other  Announce.  Pubs. 

30.70 

1.0 

30.7 

Other 

0 

3.2 

0 

Access 

5.60 

12.4 

69.4 

Reading 

28.60 

12.4 

354.6 

TOTAL 

$  44.80 

12.4 

$555.8 

V 


Source:  King  Research,  Inc. 


his  representative)  for  searching  and  reading.  In  the  case  of  TAB  the 
total  expenditures  is  $74.9  million  (i.e.,  1.5  million  readings  from 
searches  times  $49.90  per  reading).  If  a  substitute  method  of  searching 
costs  more  then,  in  order  to  maintain  the  $74.9  million  fixed  expenditures, 
fewer  readings  would  be  possible.  In  this  instance  there  would  be  about 
cne  million  readings  instead  of  1.5  million  readings  from  TAB  searches. 
The  value  lost  in  terms  of  the  scientist's  or  engineer's  willingness  to  pay 
would  be  about  $25  million.  Similarly,  the  total  amount  of  CAB 
expenditures  is  $31.4  million  and  the  number  of  searches  possible  at  $75.40 
would  be  about  400  thousand  instead  of  700  thousand.  Thus,  the  value  lost 
in  this  instance  would  be  about  $13  million .  Finally,  the  total  value  of 
bibliographic  searching  by  computer  is  $261  million  and  the  number  of 
searches  possible  using  substitutes  is  5.1  million  instead  of  6  million  so 
that  the  value  lost  in  this  instance  is  about  $39  million.  The  total 
amount  of  value  lost  (in  terms  of  willingness  to  pay)  comes  to  about  $77 
million  for  the  OTIC  bibliographic  products  and  services  (TAB,  CAB  and 
computer  searches) .  These  numbers  are  all  considered  to  be  lower  bounds  on 
the  amount  of  value  achieved  through  use  of  the  OTIC  bibliographic  products 
and  services. 


In  the  survey,  respondents  were  asked  to  indicate  the  amount  of 
savings  that  were  achieved  by  them  as  a  direct  result  of  having  read  a  DoD 
technical  report  (i.e.,  their  most  recent  reading  of  a  technical  report. 
These  savings  can  be  considered  a  measure  of  the  value  of  information  to 
the  funders.  The  average  such  savings  stated  by  the  survey  respondents  was 
$4,700  per  reading;  where  about  85  percent  of  the  readings  of  technical 
reports  yielded  some  savings.  These  savings  involved  label  and  equipment 
as  typified  in  the  following  statements; 
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TABLE  27 


Value  of  TAB,  CAB  and  Computer  Searches  in  Terms  of  Willingness  to 
Pay  by  DoD  -Funded  Scientists  and  Engineers :  1982 

($  Millions) 


TAB 

CAB 

Computer 

Searching 

Total 

Total  Readings 

1.5  million 

0.7  million 

6.0  million 

8.2  million 

Average  Value  in  Will¬ 
ingness  to  Pay 

$  49.90 

$  44.90 

$  43.50 

$  45.70 

Value  Directly  Attribu¬ 
table  to  Product  or 
Service 

$74.9  million 

$31.4  million 

$261  million 

$367  million 

Readings  Lost  by  Sub¬ 
stituting 

500,000 

300,000 

900,000 

1,700,000 

Value  Found  by  Substi¬ 
tuting  for  Product 
or  Service 

$25  million 

$13  million 

$39  million 

$  77  million 

Source:  King  Research,  Inc. 


o  OTIC  provided  some  needed  information  on  who  was  involved 

in  a  program  and  gave  foreign  contacts. 

o  Because  of  the  nature  of  the  information,  some  data  can  be 

found  only  through  OTIC  and  therefore  is  priceless. 

o  Hie  information  gleaned  from  the  search  gave  insight  into 

something  the  researcher  had  not  thought  of  before. 
Provided  insight  into  the  problem. 

o  Gave  the  contractor  an  idea  of  what  Department  of  Defense 

was  interested  in. 

o  Saved  in  writing  and  research  time  since  someone  else  had 

tested  the  information  already. 

All  such  responses  are  listed  in  Appendix  A  of  the  report.  It  is  noted 
that  results  observed  with  Department  of  Energy  scientists  and  engineers 
were  somewhat  lower  on  the  average.  DOB  scientists  and  engineers  indicated 
an  average  savings  of  about  §1,280.  They  reported  savings  for  about  75 
percent  of  the  readings  of  technical  reports. 

Hie  total  value  to  funders,  measured  in  terms  of  savings  achieved 
through  the  12.4  million  readings  of  DoD  technical  reports,  is  estimated  to 
be  about  $58  billion.  The  amount  of  this  attributable  to  technical  reports 
read  as  a  result  of  using  TAB,  CAB  and  bibliographic  computer  searches  is 
about  $37.5  billion.  Hie  value  of  savings  to  funders  found  by  substituting 
bibliographic  products  and  services  for  TAB,  CAB  and  bibliographic  computer 
services  is  estimated  to  be  about  $8.3  billion.  These  data  are  summarized 
in  Table  28.  Like  all  of  the  estinates  of  value  derived  in  this  Chapter, 
these  figures  represent  value  realized  in  a  particular  year  (1982)  from  use 


TABLE  28 


Value  of  TAB,  GAB  and  Computer  Searches  in  Terms  of  Savings  to  Funders  by 
DoD-Funded  Scientists  and  Engineers :  1982 


TAB 

CAB 

Computer 

Searching 

Total 

Total  Readings 

1.5  million 

0.7  million 

6.0  million 

8.2  million 

Average  Value  in  Savings 

$  5,600 

$  5,500 

$  4,200 

$  4,570 

Value  Attributable  to 
Product  or  Service 

$  8.4  billion  $  3.9  billion 

$25.2  billion 

$37.5  billion 

Readings  Inst  by  Sub¬ 
stitution 

500,000 

300,000 

900,000 

1,700,000 

Value  Found  by  Substi¬ 
tuting  for  Produce 
or  Service 

$  2.8  billion  $  1.7  billion 

$  3.8  billion 

$8.3  billion 

Source:  King  Research,  Inc. 


of  the  total  DTIC  data  base,  that  is,  the  cumulation  of  the  reports  and 
bibliographic  records  processed  over  the  years.  Thus  the  investment 
necessary  to  generate  these  levels  of  value  includes  both  1982  expenditures 
and  earlier  expenditures  on  collecting  and  processing  the  OTIC  collection. 

The  Defense  Technical  Information  Center  also  provides  information 
from  management  data  banks  as  mentioned  in  Sections  2  and  3.  Relative  to 
the  completeness  and  usefulness  of  the  data  in  the  data  banks,  users  were 
asked  to  place  a  value  on  staff  time  and  equipment  and  supply  costs  saved 
by  using  the  DTIC  management  databanks  as  shown  in  Table  29.  Eighty-cne 
percent  of  WUIS  users  found  they  saved  money  by  using  the  databanks  shewing 
an  average  of  $2,600  per  user  per  search.  Eighty-eight  percent  of  R&DPP 
users  saved  an  average  of  $3,600  per  user  per  search  and  finally,  74  per¬ 
cent  of  the  IR&D  users  saved  $3,900  on  the  average. 

Overall,  9  percent  of  the  survey  respondents  used  management  infor¬ 
mation  reports.  It  i_  estimated,  therefore,  that  there  are  approximately 
14,000  uses  in  total  at  an  average  savings  of  $2,900  per  search.  Hie  total 
yearly  savings  resulting  from  management  data  bank  use  is  thus  $73  million. 


To  this  point,  the  value  associated  with  technical  reports  has  been 
discussed  for  all  DoD  reports,  regardless  of  their  classification  level 
(classified,  unclassified/limited,  or  unclassified/unlimited) .  An  im¬ 
portant  issue,  however,  is  the  extent  that  classification  has  on  extent  of 
use  and  hence  on  value. 

In  Section  3  it  was  indicated  that  3,169  titles  in  FY  1982  were 
classified,  8,048  were  unclassified  but  limited  and  16,089  were  unclassi¬ 
fied  and  unlimited.  However,  the  number  of  readings  in  1982  of  classified 
technical  reports  was  2.6  million  readings  conpared  to  2.6  million  for 
unclassified/limited  and  7.2  million  unclassified  and  unlimited.  This 
means  that  the  average  number  of  readings  per  title  (recently  announced)  is 
about  twice  as  high  for  classified  reports  (820)  as  unclassified/limited 
(323)  and  unclassified/ unlimited  (448) .  Even  though  the  average  number  of 


TABLE  29 


Average  Value  in  Terms  of  Savings  of  DTIC  Management  Data  Bank  Uses:  1982 


Data  Bank 

Proportion 

With  Savings 
(%) 

Average 

$  Savings  Value 

R&T  Work  Unit 

81 

$2,600 

BSD  Program  Planning 

88 

3,600 

Independent  B&D 

74 

3,900 

Total 

82 

$2,900 

Source:  King  Research,  Inc. 
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readings  per  title  may  be  higher  because  many  of  the  2.6  million  readings 
of  classified  technical  reports  were  of  older  titles  (i.e.,  older  than 
1982) ,  the  data  do  suggest  that  the  amount  of  reading  of  classified  reports 
is  greater  than  that  of  unclassified  ones.  lhus,  the  value  of  classified 
reports  in  terms  of  willingness  to  pay  and  in  savings  is  also  likely  to  be 
greater.  In  fact,  it  is  found  that  the  average  value  in  terms  of  savings 
is  substantially  higher  for  reading  classified  technical  reports  than 
unclassified/limited  or  unclassified/unlimited  reports  as  shown  in  Table 
30. 

A  further,  and  perhaps,  even  more  important  question  is  how  much 
value  is  lost  because  of  the  constraints  imposed  on  classified  technical 
report  distribution  and  resultant  loss  of  readings.  This  question  cannot 
be  answered  directly,  although  seme  evidence  can  be  shown.  It  is  found 
that  nearly  all  readings  of  classified  technical  reports  come  from  reports 
that  are  identified  by  OTIC  bibliographic  products  or  services,  whereas 
about  one-third  of  the  readings  of  unclassified  and  unlimited  technical 
reports  are  identified  by  other  means.  This  suggests  that  an  additional 
(at  least)  one-third  readings  would  be  made  of  classified  technical  reports 
if  there  was  greater  awareness  of  them.  If  this  assumption  is  correct, 
there  would  be  about  one  million  additional  readings  of  classified  techni¬ 
cal  reports  at  an  assumed  average  value  in  terms  of  savings  of  $8,600.  Ibis 
comes  to  an  additional  $8.6  billion  in  value  attributable  to  classified 
technical  reports  or  $5.1  billion  if  the  readings  of  classified  technical 
reports  replaced  the  readings  of  unclassified  and  unlimited  technical 
reports.  This  analysis,  while  based  on  a  number  of  assumptions ,  does 
support  substantial  value  may  be  lost  by  re  tricted  announcement  and  dis¬ 
tribution  of  classified  technical  reports.  Phrased  in  another  way,  poten¬ 
tial  losses  in  reading  and  value  are  costs  which  must  be  balanced  against 
the  other  kinds  of  benefits  which  accrue  from  classification  of  DoD 
reports . 


TABLE  30 


Average  Value  in  Terms  of  Savings  of  DoD  Technical  Report  Readings  by 
Level  of  Report  Classification:  1982 


Report  Level  of 
Classification 

Classified 

Unclassified/  limited 

Uhclassified/unlimited 


Average  Savings 
_ Value _ 

$8,600 

2,400 

3,500 


Total 


$4,700 


Source:  Defense  Technical  Information  Center  and  King  Research,  Inc.  Survey  of 
DTIC  Use,  1982. 
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Answers  to  Open-Ehded  Questions 


Question:  "Were  the  results  useful  for  that  purpose? 

Wny?"  (See  page  C-2.) 

Some  information  2-3  years  old  gave  some  contacts  in  field. 

Provided  one  study  on  topics  and  added  to  other  information  obtained. 

Gave  some  info  he  wasn't  knowledgeable  of. 

Gave  info  contributing  to  pallets. 

Unis  organizations  needs  to  know  about  work  in  progress  and  work  completed 
to  avoid  duplications  of  work  with  other  organizations. 

Picked  out  individual  sites  for  possible  audit. 

Did  not  show  any  work  being  done  in  area.  Project  got  funded  —  got  info 
wouldn't  otherwise  have. 

I  did  find  relevant  documents. 

To  avoid  duplication  of  effort.  To  save  travel  time  and  money  and  to  help 
defend  the  NASA  budget. 

It  helped  me  avoid  duplication  of  work  effort  made  by  others.  It  helped  me 
write  a  Military  Standard  on  Logistics  Models. 

It  helped  me  avoid  duplication  of  effort  with  other  organizations  engaged 
in  similar  work. 

Ehabled  search  to  obtain  documents  on  the  subject. 

Gave  me  the  needed  info  to  begin  Law  program. 

Several  documents  were  pertinent. 

Received  info  to  pass  on  to  others. 

Gave  info  on  Ceramics  for  high  temperature  and  everyone  was  happy  about  the 
report. 

Report  abreast  of  present  technology. 

Made  manager  more  aware  of  relevant  data. 

Much  of  info  was  several  years  old. 

Did  ir  icate  a  certain  trend,  but  the  data  was  truncated  and  did  not  give 
the  latest  figures. 

Material  was  current. 

Report  useful  to  this  immediate  group  and  an  engineering  group. 

Received  needed  info  for  proposal. 

(xive  update  of  methods  and  dealt  with  problems  on  subject, 
gave  needed  info  as  to  where  to  find  other  info. 

Some  items  didn't  show  up,  but  was  partially  useful;  dich’t  use  all  key 
words. 

Now  I  only  have  one  piece  of  paper  instead  of  multiple  copies  or  pieces. 
Accessions  were  applicable  to  our  project. 

Called  author  and  got  report,  which  is  useful. 

Helped  to  identify  the  ongoing  work  and  other  agencies  also  working  on 
same  subject. 

Negative,  found  nothing. 

Wbrk  with  vague  out  of  date  and  inaccurate  not  complete. 

Except  time  it  takes  to  obtain  follow  on  reports  from  OTIC.  Can  call 
author . 


Wiat  yew  do-assessment  of  resea  rch-apply  to  Public  Law  9648. 

Wrote  an  IOM  on  subject  as  a  result  of  the  search. 

Other  obligation  presented. 

Question  of  no  info.  Perhaps  definition  of  terms  was  inconplete. 

Useful,  but  much  he  repeated  with  parameter  search. 

Gave  broad  based  background,  speeds  up  research. 

Gave  significant  literature  background  for  in-house  development  and 
publication. 

Nothing  on  behavioral. 

Provides  info  that  probably  wouldn't  otherwise.  Have  to  assume  studied 
program  is  complete  and  consider  all  available  info  research. 

Ghve  broad  index  for  this  field. 

Sound  relevent  info-obtain  leads  for  use  in  project. 

We  were  getting  ready  to  do  our  own  searching  and  did  not  have  to. 

No  info  found  . 

Is  it  a  real  program?  Sane  deemed  essential,  some  non-essential-like  to 
match  what  they  can  do  with  what  services  need. 

Provided  insight  into  problem. 

Some  programs  indicate  we  can  modify  equipment  to  make  it  of  interest  to 
the 

company. 

Provided  the  info  I  needed. 

No  info  in  data  base. 

Especially  for  future  purpose.  Info  for  lSt_  system  minimal.  Very 
helpful  for  next  system. 

Provided  what  others  were  doing-who  was  active  in  field. 

Applicable  but  request  was  cancelled. 

Very  helpful-OTIC  did  a  good  job-received  a  lot  of  info. 

Gave  info  to  group-however  some  info  old. 

Fulfilled  req. -didn't  locate  info.,  so  let  contract. 

Gave  several  studies  done  in  field. 

Found  info  on  what  others  were  doing. 

Said  what  other  people  were  doing  and  gave  foreign  contacts;  but  wanted 
more  up-to-date  info. 

Not  up-to-date. 

Provided  source  info  that  could  be  used  and  contacted. 

Sole  service  available. 

Gave  extensive  info  on  how  to  reduce  costs. 

Gave  status  of  items  being  worked  on  in  labs  and  confirmed  what  already 
knew  and  gave  additional  insight. 

Provided  needed  background  info. 

Able  to  find  some  areas  on  interest  and  where  funding  was. 

Large  listing  of  info. 

Found  info  useful  in  finding  out  what  Defense  was  interested  in. 

Gave  info  needed. 

Identified  other  companies/agencies  who  are  doing  research. 

Very  satisfied  with  OTIC  personnel-but  didn't  get  all  info  needed- 
Keywords  are  a  problem. 
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Need  more  advance  info-however  report  confirmed  this  info  wasn  't 
available. 

Good  source  for  current  research-is  combined  RDPP-WUIS  search. 

Got  otherwise  nonfindable  soource  of  info. 

Provide  useful  info  in  fulfilling  requirements  of  contracts-provide 
excellent 

background  info. 

They  were  applicable  to  the  info.  I  needed. 

Gave  up-to-date  info. 

Gave  more  tnan  needed-  but  got  info  needed. 

Confirmed  what  company  was  going  to  do-  very  supportive. 

Because  there  is  very  little  studies  available  on  Lantim. 

Alot  of  background  from  others'  research. 

Knowledge  of  related  industry  studies  and  gives  leads  to  contact. 

Because  it  outlined  work  done  by  others  and  they  used  it  to  analyze  their 
program. 

Gave  several  conpany  contacts  and  info. 

Several  relevant  citations  were  included  in  results. 

We  are  building  a  new  theoretical  model  and  several  searches  were  received 
and  applicable. 

There  were  several  pertinent  references-was  quite  helpful. 

Yes,  the  results  were  very  useful  and  several  documents  were  relevant. 

I  was  able  to  use  the  info  I  received. 

Material  included  was  indeed  updated  info. 

We  are  working  on  a  project  and  needed  all  avail,  info. 

Dave  needed  info  on  what  was  involved  in  program-  gave  contacts-described 
program. 

Assisted  in  study. 

Gave  spiecific  info  as  to  the  contact  and  budget  info. 

It  detailed  work  being  done. 

Insight  into  something  they  had  not  thought  of. 

Too  broad  scope. 

Received  sufficient  background  data  for  her  report. 

Some  of  what  he  was  doing  had  been  done  before. 

Gbve  necessary  history  of  present  torpedo  work  and  appreciation  for  what  is 
going  on.  Don't  know-hasn't  finished  reading  and  collecting  data. 

Found  references  I  would  not  have  found  otherwise-other  contacts  were  made 
available  to  me. 

I  got  seme  info  that  was  useful. 

A  lot  of  leads-  but  no  documents  that  were  ordered  ever  came. 

Data  seemed  insufficient  for  the  search  in  question-Data  was  used  in  other 
efforts. 

There  were  some  relevant  cites  in  the  document. 

"I  did  not  receive  any  pertinent  info  from  OTIC." 

Pertinent  Info. 

Helped  link  other  info  concerning  state  of  the  art  in  Millimeter  W Save 
Technology. 

Any  bit  of  info  is  useful. 

Several  pertinent  searches  were  received. 

Stnall  amount  available. 
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Pertinent  References. 

Pound  reports  he  hopes  will  be  useful.  Has  not  had  time  to  fully  study. 
Because  it  was  pertinent  to  my  requirements. 

He  was  disappointed  in  the  snail  amount  of  material. 

Helpful  in  providing  what  info  was  available. 

We  found  that  little  was  being  done  in  the  subject -need  to  do  more. 
Identified  number  of  similar  R&D  efforts- 
showed  work  not  being  done  elsewhere. 

Gave  knowledge  of  other  power  sources  for  army  study. 

Gave  needed  background  info. 


Question:  "Could  you  describe  how  it  saved  you  time?" 

Would  have  had  to  perform  search  on  own. 

Would  have  had  to  get  library  to  search  other  sources. 

Would  have  had  to  get  library  to  search  other  sources. 

Would  have  had  to  perform  search  in  other  areas-i.e.  library,  other 
systems. 

Manual  search  -  UCLA  had  ML 7  saves  some  time;  1/2  hour  versus  days  time. 
Would  have  had  to  perform  library  searches  &  contact  personnel  to  gather 
info.  Only  used  report  as  back-up  &  no  significant  savings  resulted  - 
probably  wouldn't  have  searched  info  if  not  available  through  OTIC  TRs. 
Had  to  repeat  research  with  all  its  mistakes  to  find  the  right  answer. 

Very  difficult  to  find  their  type  of  work  through  any  other  source. 

They  haven't  saved  time  but  has  added  to  his  research  effort. 

Also  searches  STAR,  stlil  has  to  find  out  material  from  other  sources.  Gets 
a  lot  of  data  by  word  of  mouth. 

Saved  them  from  constructing  research  that  was  done  before. 

Would  have  to  search  for  info  elsewhere  &  some  info  not  available  anywhere 
besides  OTIC. 

Would  have  to  search  info  elsewhere. 

Had  to  spend  time  looking  elsewhere. 

Ws  would  have  to  spend  time  travelling  to  other  areas  plus  searching  for 
the  info. 

Relieves  us  from  lots  of  searching  elsewehere. 

Many  hours  of  hand  searching. 

Searching  through  the  library. 

Many  hours  digging  manually. 

Time  searching  manually. 

Would  have  had  to  make  a  thorough  literature  search. 

Don't  know. 

Needs  material  within  1  to  2  days-Most  OTIC  materials  arrive  too  late  to  be 
of  use  except  in  confirming  their  findings  to  a  problem. 
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Pto le  to  respond  quickly  to  corporate  headquarters  in  LaJolla,  California, 
fenually  searches  indexes  in  open  literature.  Allows  good  background 
material  which  they  would  not  have  had  time  for  otherwise. 

Saved  literature  searching  time. 

Able  to  respond  faster  to  needs  of  our  research  group. 

Turn-around  time  is  faster  using  DTIC. 

Make  decision  of  info  needed,  then  go  out  to  recreate  info,  after  search 
was  made  for  other  sources. 

Saved  time  by  not  having  to  recreate  data. 

This  was  ordered  for  a  potential  contract.  If  they  get  the  contract  for 
developing  a  product,  they  could  realize  savings  of  $100,000  tovrard  that 
development . 

Put  subject  in  direct  contact  with  needed  info-settle  a  technical  issue. 

No  savings  compared  to  the  terminal,  but  gives  me  time  to  browse  &  gave  a 
broader  outlook. 

Material  could  not  be  found  elsewhere  so  fast. 

Saved  literature  searching  time  when  DROLS  computer  terminal  was  down. 
Turnaround  time  is  fasdter  using  DTIC. 

Results  are  more  complete  and  data  is  more  quickly  accessed  than  elsewhere. 
Saved  literature  searching  time. 

TAB  used  as  backup  when  DROLS  computer  down. 

Turnaround  time  is  faster  using  DTIC  service.  Provided  info  that  we  didn't 
currently  have. 

Finding  reports  in  TAB  saves  time-  Reduces  the  number  of  places  to  look. 
Over  the  course  of  a  project,  the  documents  involved  can  save  $15,000  on 
$3,500,000  project. 

Gave  additional  references  to  P _  Info. 

Did  not  need  to  order  these  reports.  Already  knew  data  contained  in  these 
reports.  Normally  it  does  give  new  info. 

By  not  having  to  spend  time  doing  library  research,  etc. 

In  doing  my  own  travelling  and  searching. 

Researching  through  library. 

Not  having  to  repeat  the  research  that  was  done. 

Planning  time  and  time  in  developing  procedures. 

Did  not  have  to  leave  work  area  for  research. 

Research  time. 

Discovered  that  the  work  had  not  been  done.  Beginning  of  a  new  project. 

It  was  negative  infornvation/still  alot  of  work  to  be  done  on  this  project. 
We  were  able  to  get  the  information. 

Not  a  technical  report  in  specific,  but  a  assessment  of  technology-not 
possible  to  put  a  dollar  value  on. 

Sole  source  of  information-only  place  to  get  it. 

Went  to  other  documents  for  the  particular  information. 

TAB  is  only  source  of  information. 

Did  not  have  to  repeat  research.  Had  a  reliable  source  of  information. 

It  gives  one  a  better  view  of  the  problem.  Documents  found  quickly. 

Don't  have  to  go  through  all  journals.  Don't  have  to  go  through  companies. 
Time  spent  going  elsewhere,  and  checking  with  other  colleagues. 

Did  not  have  to  duplicate  research.  Data  was  there. 


Don’t  have  to  do  mechanical  search,  but  at  this  time  it  does  not  seem  that 
survey  is  complete.  Needs  to  change  search  strategy. 

Not  address  the  subject. 

Would  have  had  to  go  to  library  and  search  for  information. 

Would  have  had  to  perform  own  research. 


One  centralized  source  for  much  information. 

Two  weeks  of  computational  work. 

More  research  needed. 

Would  have  had  to  "Re-invent  the  wheel" . 

I  got  it  pronptly  when  I  needed  it  most. 

The  entire  system  was  down  and  I  needed  a  quick  response. 

1  did  not  have  to  look  through  all  literature  sources  to  find  what  I 
wanted. 

We  might  have  lost  out  on  an  RFP  of  we  hadn't  gotten  the  information 
promptly. 

The  computerized  on-line  system  was  not  operating  and  I  needed  to  reach 
promptly. 

We  did  not  duplicate  listing  and  research  which  had  already  been  done. 

It  saved  my  having  to  search  all  the  literature. 

Reduced  research  time,  freeing  me  for  other  tasks. 

Had  to  have  done  research  himself. 

Make  more  phone  calls. 

Time  spent  not  having  to  do  research. 

Two-Bnree  days  of  work  saved  not  having  to  do  the  search  elsewhere. 

Would  have  had  a  lower  quality  product  because  he  would  no  have  had  this 
info  .  Since  most  of  what  he  uses  is  classified,  would  have  been 
difficult,  if  not  impossible,  to  get  info  elsewhere.  By  having  DTIC's 
reports  available,  didn't  have  to  go  elsewhere  to  find  info. 

Didn't  have  to  go  elsewhere  to  find  info. 

Library  would  have  to  search. 

Know  about  info  that  wouldn't  have  known  about  otherwise-  may  not  have  had 
the  info  available. 

Would  have  had  to  contact  individuals  in  this  area  and  have  discussions 
with  them. 

Time  and  travel  to  do  research  by  other  methods. 

Telephone  interviews  and  traveling. 

Would  have  had  to  look  for  other  bibs,  and  libraries  and  spend  time  doing 
personal  research. 

Was  able  to  see  methodology  that  could  be  applied-  didn't  waste  time  using 

methodology  that  didn't  work. 

Time  saved  in  having  terminal  instead  of  going  to  DTIC  with  key  words. 

Don't  know  yet  how  reading  reports  saved  time. 
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Would  not  have  had  most  of  info-ha  s  not  been  successful  in  locating 
materials  in  past. 

Would  have  called  agencies  if  knew  who  to  contact-DTIC  is  irreplaceable 
OTIC  is  necessary  to  his  work -could  not  be  duplicated  therefore. 

Travel  time  for  moving  pice  to  place  &  time  for  searching. 

Searching. 

We  were  relieved  of  doing  our  own  searching. 

We  would  have  had  to  do  our  own  searches. 

Our  time  was  saved  by  someone  else  doing  the  looking. 

Time  spent  searching. 

Staff  did  not  have  to  devote  time  to  searching., 

Not  having  to  do  our  own  searching. 

We  would  have  spent  several  hours  doing  searches. 

Not  having  to  do  our  own  research  for  revelant  subject  matter. 

Travel  &  not  having  to  do  our  own  searching  through  volumes  of  other 
material. 

Doing  research. 

We  would  have  had  3  or  4  other  people  working,  searching  for  these  reports. 
By  not  having  to  do  our  own  searching. 

Hie  last  report  one  read  was  not  applicable,  however  when  they  are,  I  save 
time  by  not  having  to  look  for  info  elsewhere. 

Would  have  had  to  look  through  other  references-textbooks  other  info. 

Would  have  had  to  write  a  computer  program  to  obtain  the  analysis  after 
digging  up  the  material. 

Finds  work  already  done  and  people  to  contact. 

Relevancy  90%  vs.  0%  -  1%.  Keep  up  a  leading  edge  of  state-of-the-art. 
Would  have  called  government  agencies. . 

I  read  about  it  and  was  able  to  identify  it  promptly. 

Would  have  developed  it  or  gone  without-probably  the  latter. 

Would  have  researched  other  sources  such  as  NTIS,  chemical  societies. 

Would  have  used  other  data  bases  or  manual  search  of  literature. 

(Hasn't  received  items  yet). 

Would  have  had  to  do  alot  of  basic  work,  with  uncertainty  and  risk  in 
securing  findings. 

Added  to  the  base  of  knowledge  for  this  project. 

Speeds  up  time  in  finding  materials. 

We  were  able  to  respond  promptly  to  a  DOD  request  for  proposal. 

It  saved  search  etime  response  becasue  our  computer  terminal  was  not 
operational. 

We  were  able  to  obtain  contracting  and  subcontracting  leads. 

The  data  was  not  readily  availabl  eelsewhere. 

Information  was  readily  available  to  take  advantage  of  a  DOD  procurement 
invitation 

Saved  searching  time. 


It  is  not  readily  available  elsewhere.  We  could  not  operate  so  well 
without  one  products  Keep  up  with  the  state-of-the-art  and  can  work 
more  effectively. 

Would  have  contacted  people  in  field  for  reports. 

would  have  done  own  search  or  performed  library  search. 

No  other  way  to  get  the  information. 

A  void  in  this  particular  area  saved  time  and  transit. 

It  helped  us  react  pronptly  in  preparation  of  a  research  proposal. 

We  would  have  to  duplicate  test  results  and  we  were  able  to  decide  that  we 
also  wanted  the  classified  reports. 

Saved  time  having  to  look  elsewhere. 

Would  have  had  library  search  from  other  sources. 

Library  would  have  done  searches  from  other  sources. 

Would  have  completed  job  on  own  knowledge/experience  and  perhaps  go  to 
experts  in  field. 

Would  have  completed  on  own  knowledge/ experinece  aknd  perhaps  go  to  experts 
in  field. 

Would  not  have  gotten  the  info. 

Would  have  gone  to  library  to  secure  some  of  the  info. 

Would  have  gone  to  the  library  to  secure  some  of  the  info. 

Would  have  used  other  indexes-  would  have  taken  the  same  amount  of  time 
that  OTIC  does,  however  he  prefers  our  service  because  it  gives  him  more 
info. 

Would  have  used  other  indexes-  would  have  the  same  amount  of  time  that  OTIC 
does;  however,  OTIC  gives  more  info. 

Would  not  have  had  access  to  info. 

Search  time. 

We  did  not  have  to  spend  our  time  searching. 

By  having  OTIC  identify  relevant  abstracts. 

The  searching  for  relevant  documents. 

By  not  having  to  do  our  own  searches. 

Telephone  calls,  direct  contact,  etc. 

Trips,  telephone  calls,  etc. 

Time  saved  in  getting  document  easily. 

Look  at  CAB  at  getting  something  extra;  not  as  a  savings-  would  not  get 
material  otherwise. 

Did  not  have  to  go  to  nany  other  soruces  to  get  the  same  info. 

The  CAB  saves  the  manual  work  offuinding  the  documents,  may  of  hwichcould 
be  missed. 

Need  to  find  info,  elsewhere. 

This  research  is  done  as  if  a  personal  SDI-is  the  only  way  he  searches- 

Does  not  have  idea  of  how  long  it  would  take  to  get  this  info  in  other 
manners. 

Collection  of  data  that  would  have  had  to  have  been  gathered  from  may  other 
sources. 

Do  not  have  to  write  to  research  groups  to  get  the  info. 

Need  to  have  gone  back  to  the  source  study. 

Getting  naterial  eliminates  reinventing  the  wheel. 
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Having  automated  searching  speeds  up  finding  the  documents. 

Would  have  gone  to  organization  who  issued  report. 

Would  have  generated  info  himself. 

Would  have  canvassed  other  services  for  info ./reports. 

Would  have  used  other  databases/ reports-he  uses  broad  resources. 

Would  have  contacted  individual  people. 

Would  have  used  other  search  methods  in  library 

Survey  type  report -lot  of  research  bibs-did  alot  of  initial  sorting. 

Done  for  a  special  project. 

Have  to  get  data  from  other  sources-report  quite  comprehensive. 

Documents  used  to  complete  fact  sheet  on  data  needed  in  24  hours.  Would 
have  taken  much  longer  to  find  data  wanted  can  short  notice. 

Have  access  to  info  so  research  was  not  repeated. 

Showed  that  the  work  had  not  been  done. 

Someone  else  has  developed  a  concept. 

Time  saved  in  reduction  of  searching. 

Hake  us  aware  of  current  trends. 

Would  have  contacted  Federal  agencies  and  contractor s-but  wouldn't  have 
equivalent  info. 

Would  have  called  Federal  agancies  and/or  contacted  contractors. 

Info  was  delivered  promptly  by  library  so  that  briefing  was  prepared  can 
time. 

Material  was  not  available  elsewhere  and  were  very'  specific. 

Titles  were  available  promptly  for  my  briefing. 

Material  was  probably  available.  Saves  time  designing  instructional 
materials. 

We  did  not  have  to  provide  secondary  distribution  of  our  own  handbook. 

Saved  search  time  becasue  DROLS  computer  was  down. 

Brought  us  up  to  date  on  current  concepts. 

Background  material. 

Time  saving  in  searching  through  TAB-critical  for  doing  research-  However, 
TAB  is  not  very  complete  on  the  Behavioral  Scieinces.  Feeling  that  it 
is  an  excellent  service. 

Would  have  made  personal  contacts  in  his  field  of  interest. 

Would  have  gone  to  author  to  secure  report. 

Would  have  gotten  a  contractor  to  secure  info. 

Would  have  used  private  databases  or  gone  to  universities. 
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Would  have  performed  studies  personaly. 

Hasn't  received  reports. 

Would  have  contacted  authors  after  lengthy  search  for  report  titles. 

Did  not  have  literature  searches. 

Time  saved  by  getting  reports-  No  idea  of  how  long  it  would  take  to  get 
reports  otherwise. 

Do  method  of  evaluating  time  saved.  Still  in  training. 

Turnaround  time  is  faster  using  DTIC  products. 

Saved  time  because  DHOLS  computer  terminal  was  not  operating. 

We  were  able  to  respond  promptly  to  the  researchers  request. 

Turnaround  time  is  faster  using  DTIC  products  and  services. 

We  are  six  to  eight  weeks  ahead  of  schedule  using  CAB. 

We  could  not  have  found  it  elsewere. 

Saved  time  because  the  computer  terminal  was  down. 

Turnaround  time  is  faster  using  DTIC  services. 

Turnaround  time  is  faster  for  us  using  DTIC. 

Turnaround  time  is  faster  for  us  using  DTIC. 

Would  have ' reinvented  the  wheel" -contacted  people  in  field  of  interest. 
Reduced  time  involved-enabled  to  get  info  quickly. 

Narrowing  down  the  search-  Having  to  use  other  sources. 

Gives  outside  contacts  for  further  info. 

Would  have  called  other  activities  involved  in  sane  area  of  interest  to  get 
reports. 

Would  have  called  agencies  for  info. 

Would  have  searched  for  people  in  the  field  to  speak  with. 

Would  have  to  research  different  Agencies  for  information-  cal,  travel. 

No  response. 

Expect  report  to  save  time  in  giving  much  background  information  for  future 
research. 

A  case  of  looking  for  information.  Information  is  not  available  through 
other  suorces. 

Would  have  searched  in  "open"  literature. 

Would  have  contracted  other  sources  such  as  people  he  knows  in  the  field. 
Would  have  been  "lost"  without  the  services-significant  time  to  secure  info 
from  other  sources  if  ever  possible. 

Would  have  been  "lost"  without  the  services-time  to  secure  info  from  other 
sources  if  ever  possible. 

Would  have  used  info-  not  that  critical  to  work. 

Would  have  gone  to  agency  involved  to  secure  info/report. 
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Would  have  called  other  agencies  for  reports  or  performed  research. 
Conclusion  led  than  to  drop  filers  as  a  program.  This  could  lead  to 
enormous  savings. 

Don't  have  to  do  a  alot  of  running  about  to  find  reports. 

Classroom  reading-will  probably  add  time  of  the  student. 

Better  defined  a  research  topic. 

CAB  saves  time  in  not  having  to  track  down  these  documents  (citatoins)  each 
week. 

Additional  research  material  that  was  essential  to  task.  Did  not  have  to 
recreate  data  of  report. 

Would  have  contracted  out  to  acquire  info. 

Wouldn't  have  used  that  info  in  projects. 

Would  have  gone  to  report  source  to  acquire  info  (if  known)  otherwise  would 
do  without. 

Would  have  contacted  government  agencies  of  people  in  field  of  interest. 
Would  have  contacted  government  agenies  for  info/reports. 

Would  have  contacted  a  local  university. 

Time  saved  in  method  of  accessing  report. 

Hie  reports  are  the  job-  Quickness  of  access  saves  time-2  days  work  saved 
in  not  having  to  walk  through  bib.  material  (Savings  at  l^t  Lt  at  2-4 
year  salary) . 

Would  have  conducted  interviews. 

Report  is  stlil  to  be  read-  will  require  alot  of  time. 

Would  take  up  to  a  week  of  time  to  find  the  reports  which  the  CAB  gives 
him. 

Saved  him  from  having  to  perform  the  same  work . 

This  is  saving  given  is  on  search  time  saved  by  finding  bibligraphic 
online. 

Saved  the  drudgery  of  a  difficult  manual  search. 

Would  have  not  been  aware  of  the  nifo-  would  have  performed  own  search. 
This  is  his  work. 

Depends  on  savings  of  preventing  the  need  for  future  research. 

we  were  able  ot  respond  promptly  to  our  research  staff  needs. 

This  type  of  intelligence  was  not  readily  available  elsewhere. 

Travel  time  to  do  searching-  also  spending  time  on  manual  searching. 
Identifying  from  the  abstracts  available. 

Manual  searching. 

DTIC  did  the  work. 

Implied  to  future  planning  and  professional  awareness. 

Ehables  him  to  get  needed  info  in  a  short  period  of  time.  Ehables  him  to 
be  more  productive-  get  more  info. 

Wauld  have  to  do  manual-  manual  search  probably  would  have  taken  a  month. 


Would  not  be  accessible  without  on-line  database  set  up  the  way  it  is. 

May  have  saved  someone  down  the  line  time. 

Did  not  have  to  do  manual  search. 

Did  not  have  to  duplicate  research. 

Not  having  to  duplicate  research.  Time  saved  in  identifying  additional 
references  in  each  report. 

By  not  having  to  search  elsewhere. 

Time  spent  on  researching. 

Didn't  have  to  find  the  info  elsewhere. 

Having  OTIC  do  the  research  for  us. 

By  having  the  research  done  for  us. 

By  having  OTIC  doing  the  researching,  and  travel  time  also. 

We  are  all  spoiled  by  using  the  computer-  do  not  have  to  research  in 
library. 

Travel  to  get  reports. 

Gain  info-  making  a  point. 

Haven't  gotten  data  yet,  so  can't  evaluate.  Data  in  past  has  been 
fruitful. 

This  is  for  a  gain  in  material,  not  a  savings  of  time.  Work  provided  info. 
All  work  done  by  staff  and  aides.  He  does  not  pay  any  attention  to  it. 

Did  not  have  to  recreate  processes  of  establishing  the  methods  of 
research. 

Info  readily  available. 

Did  not  have  to  repeat  research-  could  be  from  2  to  10  man-years  of  work. 
Only  place  where  the  research  is  available.  Can't  make  estimate  about  the 
time  it  takes  to  get  these  things. 

Quicker  to  do,  much  more  accurate. 

Contribution  to  project. 

Didn't  have  to  visit  the  source. 

Given  much  better  background  on  U.S.  capabilities.  How  to  direct  efforts 
toward  foreign  technologies. 

Read  to  broaden  knowledge. 

Can  find  out  what  is  done. 

Otherwise  have  to  do  less  productive  and  more  time  consuming 
resea rch-ma inly  interested  in  military  application. 

Try  to  set  up  definition  themselves,  then  get  it  approved. 

Where  to  find  available  info. 

Add  value  to  the  project. 

Did  not  have  to  spend  time  that  a  manual  search  would  have  taken. 

Did  not  have  to  recreate  research. 

Design  our  survey  and  acquire  results  to  use  on  projects. 

Some  of  the  info  would  not  be  directly  available  and  would  have  to  contact 
other  organizations  for  info. 

Would  have  had  to  call  other  organizaitons  for  information/materials. 
Research  field  of  study  &  probably  result  in  duplicate  effort.  Wouldn't 
have  the  opportunity  to  build  on  someone  else's  work. 

Would  have  had  to  go  to  the  Records  center  to  search  records  for  answer. 
Have  to  go  to  library  and  research  and  review  articles. 

Call  originating  agency-  how  to  obtain. 

Would  have  a  lower  level  of  technology  and  be  ignorant  of  info  in  this- 
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would  have  had  to  perform  our  research  but  would  not  have  obtained  total 
info. 

Go  to  specialist  at  Lockheed-This  would  have  reuqired  a  trip. 

Items  to  be  found  at  one  spot. 

Not  a  savings  of  time  but  a  gain  of  information. 

This  was  an  initial  search  to  see  if  info  on  abstracts  would  show  problems 
of  finding  things,  as  current  key  words  did  not  match  the  aproved  DRIT 
list. 

In  conjunction  with  another  report  could  respond  to  a  Logistics  need 

Summary  without  survey  of  20  other  reports. 

Did  not  have  to  contract  search  out. 

Value  given  to  the  providing  of  info.  Material  not  known  to  exist 
elsewhere. 

Without  research  aids  such  as  this,  valuable  data  would  be  missed. 

Haven't  done  reading.  Can  save  time  only  if  another  has  something  new  to 
offer. 

Conputer  does  the  work.  Need  to  repeat  research  and  look  at  a  lot  of  other 
places . 

Did  not  have  to  go  elsewhere-  data  easily  provided.  Make  more  efficient 
use  of  time. 

Had  a  read  access  to  the  material. 

Item  used  to  keep  aware  of  current  state  of  research  in  the  field. 

Still  examining  materials  to  begin  project. 

Did  not  have  to  reconstruct  a  war  game. 

TAB  is  a  source-  without  it  he  would  not  know  work  exists. 

Although  more  work  has  resulted,  time  was  saved  by  not  having  to  search  for 
materials. 

Would  have  had  to  go  to  Washington,  D.C.  and  perform,  search  and  would  take 
2-3  days. 

Would  have  had  to  go  to  library  and  perform  search  of  materials. 

Not  read  yet. 

Would  have  had  to  duplicate  most  of  repots  on  research  library  for  ohter 
sources  of  info. 

Not  having  to  do  the  research. 

Do  our  own  research. 

Reading  was  a  gain  of  data;  used  to  find  other  reports  from  bibiliography. 

Did  not  have  to  make  trips  to  get  documents.  Bibliographies  there  to  check 
off  against  current  holdings. 

Would  have  to  go  to  other  source  to  get  information  through  commercial 
access-  more  time  consuming,  more  expensive. 

Did  not  have  to  duplicate  research. 

TAB  acts  as  a  good  source  of  info. 

Wouldn't  have  gotten  all  the  info  but  would  have  searched  in  library. 

Spend  time  in  library  searching  for  information. 

Go  to  library  or  other  suorces  to  find  info. 

Go  to  other  sources  for  info. 
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There  is  a  much  greater  speed  of  researching  on  our  data  team. 

Would  have  used  ERIC  data  base. 

Provided  extra  info  for  the  project  on  hand.  Did  not  have  to  recreate  a 
search. 

Documents  found  at  a  single  location. 

Wouldn't  have  known  about  reports-  because  they're  limited-  probably 
wouldn't  have  info-  but  would  do  sane  library  research. 

It  gives  him  a  convenient,  quick  summary  of  what  is  current,  but  he  insists 
it  does  not  save  him  time  or  nmoney. 

Saved  literature  searching  time. 

Material  could  not  be  found  elsewhere. 

Saved  literature  searching  time. 

Spend  the  hours  of  basic  research  plus  put  it  together. 

Short  time  to  get  classified  info. 

There  were  delays  in  the  initial  ordering  process.  Report  enabled  this 
research  to  be  up  to  date  and  not  repeat  research.  Can  also  learn 
successful  techniques. 

May  have  saved  1  say -possibly  o  weeks  time. 


Question:  "What  did  you  and  your  staff  save  as  a  result  of  that 

reading?  (See  pages  C-7,  C-8.) 


Readily  available  information. 

Time  from  looking  for  other  sources 
Time 

Spend  time  learning  about  answers  to  particular  problems. 

Don't  know  six  months  of  trial  and  error. 

Not  yet  arrived-  will  save  time  if  they  live  up  to  their  abstract 
Did't  have  to  do  the  work  described  in  report. 

Time. 

Time. 

Time. 

Time/Money. 

Time. 

Time. 

Time. 

Time. 

Time. 

Time  and  Money. 

Time  and  Money. 

Time/Money 
Time/Money . 

4  days  of  Effort. 

2  days  on  each. 
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Answered  a  question. 

Time/Money. 

Time  and  Money. 

Time  and  Money. 

Locating  information. 

"Why  do  you  ask  these  questions-  it  is  an  obvious  consideration  that  any 
very  recent  search  would  not  yet  have  produced  documents.  Also  are  we 
placing  value  on  the  BIB  or  on  the  documents  themselves." 

Got  more  info. 

Time. 

0. 

Time/Money. 

Time. 

Time/Money. 

Time/Money . 

Planning  time  and  Time  in  Deveoping  procedures. 

Time  and  Money. 


Nothing. 


Used  in  preparing  Handbook. 

Time. 

Saved  "Re-inventing  the  wheel" . 

Time  saved. 

Time. 

Time-  did  not  have  to  "re-invent  the  wheel". 

Time  &  Money. 

One  hour. 

Manhours  and  Money. 

One  hour. 

Time  and  money. 

$30. 

Completeness. 

Hours  of  time. 

Got  results  from  what  others  have  already  done. 

By  having  DTIC's  reports  available,  didn't  have  to  go  elsewhere  to  find 
info. 

Didn't  have  to  go  elsewhere  to  find  info. 

Library  would  have  to  search. 

G&ined  access  to  something  they  didn't  know  about. 

Would  have  had  to  contact  individuals  in  this  area  and  have  discussions 
with  them. 

Time. 

Time. 

40  hours. 


Saved  personal  time  in  searching  info  in  library. 

Mas  able  to  see  methodology  that  could  be  applied-  didn't  waste  time  usinq 
methodology  that  didn't  work. 

Could  not  get  info  otherwise. 

Time. 

Time. 

Time  and  Money. 

Time  and  Money . 

Time. 

Time. 

Time  and  Money. 

Time  and  Money. 

Time  and  Money. 

Time  and  Money. 

Time  and  Money. 

Could  do  other  things;  one  week  of  searching  elsewhere. 

Plagarize  Analysis. 

Work  already  done-  have  to  meet. 

Time. 

Mon ey//Manhou r s  Time/Cost. 

Time. 

Time. 

Better  quality  reports. 

Time  and  Money. 

Time  and  Money. 

Time  and  Money. 

Time  and  Money. 

Time  and  Money . 

Time. 

Time. 

Invaluable — no  other  source. 

Time  and  Money. 

Time  and  Money. 

No  answer. 

Time  and  Money 
Time  and  Money. 

Confirmed  prior  knowledge. 

Time. 

Time. 

Time. 

Time. 

Time. 

Time. 

Time  and  Money. 
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I 

Time. 

Time. 

Personal  Time. 

Time/Cost. 

Time/Cost . 

Saves  patrons  time. 

Cost. 

Time. 

Cost-  Too  lengthy  time, 
feve  answer. 

Time  and  Money. 

Time. 

Got  info  probably  wouldn't  have  acess  to. 

Time  and  Money. 

Time  and  Money. 

Time  and  Honey. 

Time. 

Time. 

Time. 

Time. 

Time. 

Time/cost 

Time. 

"This  is  the  same  result  as  for  Part  I"  (no  result  given  in  part  I.) 
Time  and  Money. 

Time  and  Money. 

Time  and  Money. 

Time  and  Money. 

Time  and  Money. 

Time  and  Money. 

Time  and  Money. 

Cost. 

Cost. 

Reduced  time  involved-  Enabled  to  get  info  faster. 

Time. 

Time. 

Time. 

Time. 

Time. 

Time/Costs. 


Time. 

Time. 

Cost. 

Personal  interest. 

Time. 

Time. 

Cost  and  time  to  do  our  own 
Cost  and  time  to  do  our  own 


research. 

research. 
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Using  TAB  saves  time  in  finding  documents. 
Cost/Time. 

Had  info  readily  available. 

Better  effectiveness. 

Better  effectiveness. 

Time. 

Time. 

Time. 

Time. 

Time/Cost . 

Cost. 

Time  and  Money. 

Time. 


Time  and  Money. 

Time. 

Article  Screened  ahead  ofTime. 

Tremendous  amount  of  Time. 

Time. 

Develop,  development  of  computer  models. 

Saved  time. 

Time. 

Time. 

Time. 

Reliabiltiy  and  credibility. 

Got  graphs  of  VS  technologies. 

Bidless  hours  of  figuring  things  out  when  the  material  is  available  in 
literature  2  years  of  work. 

Not  available  if  not  found  here. 

Better  job  with  info. 

Good  info. 

Familiar  with  areas  of  interest. 

Made  info  available  for  instructional  purposes. 

Design  our  survey  and  acquire  results  ot  use  on  projects. 

Time  Explicate  effort. 

Several  months. 

Time. 

2  to  about  20,000. 

Time . 


Information. 

0. 

5%  of  effort  that  would  be  expended  elsewhere. 

Presented  a  good  simulation-  saved  several  months. 

Although  morr  work  has  resulted,  time  was  saved  by  not  having  to  search  for 
materials. 

Actions/projects  have  resulted  from  reading  report. 

Reports  were  used  for  brewsing  only  and  savings  haven’t  been  realized  yet. 
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Time. 

Not  read  yet. 

Time. 

Gives  info. 

Cuts  down  or  second  guessing-better  factual  basis  for  making  decisions. 
Time. 

Gbve  info  to  complete  projects-saved  time. 

Time. 

Hade  organization  more  efficieint-time  saved. 

Time. 

Avoided  duplicate  effort  and  used  their  recommendations. 

Time/Money. 

Time/Honey. 

Research  time. 

Indispensable,  essential  access  in  short  time  work  being  done. 


Question:  We  are  interested  in  your  use  of  reports  provided  by  OTIC, 
you  tell  us  what  report  you  read  last?  Title:  (See  C-6.) 

Anaylsis  of  future  development  witn  in  Air  Force. 

Plasticity. 

High  speed  bearings 
Don't  Know. 

Ghs  Chromotography. 

Active  Aperture. 

Development  and  Utilization  of  Integrated  and  Multi-dimensions  salary. 
Group  on  forecasting  methods  for  defense  contractors. 

Infrared  Detection  Unit. 

Surveillance:  Navy  Fleet,  Fire  Exiting.  Cartridge. 

Mechanical  Properties  of  70/50  T-73. 

Fracture  Toughness  Data  for  Disc. 

Nuclear  Fuel  Reprocessing. 

Radar  Signal  Processor. 

Vtersaw  Pact  Logistics  Planning  Guide. 

Development  of  a  Low  Cost  EMP  Protection  for  Ops.  Ctrs. 

Models  for  Warfare  at  Sea. 

Management  Engineering  Plans  for  Secure  Communications. 

Communications  Control  and  Security. 

MaltiSource  Data  Integration  for  real  time  Aircraft  ID. 

Results  of  French  Nuclear  Testing. 

Microwave  Circuit  Analysis  and  Synthesis. 

Navy  Research  and  Development  1982. 


Can 


Electrooptics  Tecnology. 

Field  Purification  of  Water  of  BX. 

Directs  Evaluation  of  al  Test  Helmut  Mounted  Site. 

Air  Base  Survivability  Workshop. 

Army  Report  an  Chemical  Warfare. 

1978  Defense  Science  Board  Survey. 

Approach  to  Test  Plan  for  Aircraft  Operation  in  a  Toxic  Environment. 
Measurement  of  Film  Cooling  Effectiveness  in  short  duration  Wind  Tunnels. 
Description  of  Major  results  of  Air  to  Ground  Visual  Recognition. 

Diffuser  Science  Board  Summary  Study  in  Technology. 

Infrared  Decoy. 

Recent  Development  in  High  Performance  Ceramics. 

Fracture  Mechanics. 

Evaluation  of  Duplex  Whiskered  Silicon  Nitride  Structure. 

Condidate  R  &  D  Thrust  for  R  &  D  Initiative. 

Projectory  Reconstruction  Progress. 

Maintaining  Ability  Verification/Demonstration  Evaluation. 

Characteristics  of  Nettle  Vapor. 

Handbook  of  Insructional  for  Non-Deductive  Testing  of  Materials. 

Flight  Safety. 

On-Line  Systems. 

NAVSTAR  Global  Post ion ing  System. 

Security, Privacy,  National  Vulnerability 
Cryogenic  Alloys. 

Antenna . 

High  Spot  Project. 

Laser  Cormunication. 

Fi refight  2  Model. 

Defense  of  ships. 

Artificial  Intelligence. 

Artificial  Intelligence. 

Sensitivity  Analyses  of  Platform  in  Target  Location. 

Reflectivitiy  Measurements  witn  10.6  Micrometer  Infrared  Radiation. 
Lasers. 

Literature  Surevey  of  Physical  and  Chemical  Properties  of  Agents. 

Military  Organization,  Readiness  and  Sustainability. 

Microwave  Radioactive. 

QUICK  Simulation  Model. 

Fiber  Optic  Couplers. 

Projectile  Characteristics. 

Aluminum  Coatings. 

Designs  for  Control  of  Projectile. 

Aluminum  Alloy  Technology. 

Environment  Storage,  Test  of  NATO  7-62. 

Tactical  Systems  in  Technology. 

Tactical  Warfare. 


A  Programmable  Wide  Band  Pseudo  Randan  Coded. 

Radar  Detection,  Analysis  for  Map  of  Earth,  Helicopter  Flight. 

A  Survey  of  Underwater  Models. 

Conmunications. 

Ship  Defense  Systems  ADT. 

Hydrazine  Detection. 

Slide  Rocket  Proponents. 

Nuclear  propulsion. 

Superplastic  forms-Aircraft 

Documenttioi  of  Software  in  the  OL-192  program. 

Very  High  Speed  Integrated  Circuits. 

Journal  of  Insrumentation  and  Control. 

H.S.I.C. 

Journal  of  Defense  Research. 

Pulsed  Laser  Effects. 

Chemistry  Reflection  change  of  a  copper  surface. 

Dive  control. 

Subminiature  Electronic  Equipment. 

CC  Intelligence. 

FFT  Algorithms. 

Microstructure-Nickel  Base  Alloy. 

Propellants-Munitions . 

Cavitation  of  Pumps. 

reparation  of  Radiation-Adhesive  Bonding. 

Aerothermodynamics . 

Investigation  of  Cold  Cathodes  for  Long  Life  CO^feve  Guide  Lasers. 
Infrared  Countermeasures.  £ 

Feasibility  of  Fuse  for  Small  Caliber. 

Air  Cushion  Vehicles. 

Electrical  Vehicles. 

Infrared  Analysis  of  Land  Combat  Vehicle  and  Land  Background. 

Effect  of  Laser  on  Army  Optics. 

Pershing  Missile. 

Interior  Ballistics. 

Pulse  Laser. 

Shaped  Charges. 

Interrelationship  of  Rock  Properties. 

Land/Mine  Warfare. 

Elastic  Scattering. 

Anti-air  Honing  Missiles. 

IDA  Report -Effect  of  Weather  Hanover  FRD  on  elect roopitcal  imaging  sys. 
3  Dimensional  Computation  -Target  Interaction. 

Cost  Analysis. 

Development  of  Archives  for  Existing  Data  Bases. 

USAF  Avionics  Plan. 
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Technology  of  space  Based  Radar. 

Analysis  of  Training  Management. 

Leadership  Criteria  for  Army  Instructors. 

Library  Handbook. 

Survey  of  Performance  Effectiveness  Measure. 

Shaped  Charges. 

Missiles. 

Laser  Effects. 

ADA  (Computer  Language) . 

Target  Vulnerability-Munitions. 

Defense  Repression. 

Environmental  Implications  of  Ground  Based  Airborne  Hydrogen  Fiouride  Laser 
Operation. 

Radar. 

Airlift  Operations. 

Library  Handbook. 

AFFIT  These  (Bibliography),  Military  Retirements. 

Mine  Countermeasures. 

Electromagnetic  Pulse. 

Linear  Shaped  Charges  Mine  Fields. 

Sunmary  Report  on  German  Research  on  Swept  Wing. 

Acoustics. 

Soviet  Import  Controls. 

Tactical  Electronics  System. 

Diving . 

Foreign  Support. 

Penetrating  Munitions. 

Status  of  Research  an  Infrared  Fiber. 

Materials  VUlnerabilty  of  OS-an  update. 

Damage  Assessment  Verification  Study. 

Vacuum  Voltage  Breakdown  as  a  Thermal  Instability  of  the  Emitting 
pultrusion. 

Circular  Polarized  Static  RCS  Patterns. 

Synthetic  Radar. 

Operations  Analysis. 

Aerothermodynamic  heating. 

Multi -band  Antennae  System. 

Natural  Inaguage  Processing. 

Conmand  and  Control. 

Joint  Acquisition  of  Weapon  Systems. 

Chemical  Warfare. 

Projection  Pursuits. 

Self-forging  Fragment  Calculation. 

(ARSV)  Armored  Reconnaisance  Scout  Vehicle. 

Magnetics. 

Fire  Control  System  for  M-l  Tank. 

Allocation  of  Military  Resources. 

Advance  in  Naval  Architecture. 
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Wiite  Phosphorous . 

Tactical  Nuclear  War  at  Sea. 

Design  Requirements  of  the  ASWS  Standoff  Weapon. 

Software  Cost  Estimating. 

Expert  System-  Newest  Branchild  of  Computer  Sciences. 

Wide  Angle  Drive  Transmission  for  Hydrocoil  Patrol  Craft. 

Distributed  Data  Processing. 

Incentive  Contracts. 

Contract  Incentive. 

NBSRffl.-23.42  Fed.  Sol.  Program  Users  Manual. 

Test  of  Satellite  Terminals. 

Dry  Oil  Battery  Usage. 

Development  of  an  All-Metal  Thick  Film  Cost  Effective. 

Metallization  for  Solar  Cells. 

Desert  water  Problems. 

Human  Performance  in  Continuous  Operation  . 

Fuel  Cells. 

Avation . 

Decision  Support  Systems. 

Cost  Growth  in  Defense  Acquisition. 

Africa. 

High  Power  Millimeter  Wave  Anpliter. 

Tank  Fire  Control. 

Multiple  Attribute  Decision  Analysis  model. 

Matrix  Management. 

List  of  Data  Sources. 

Program  Management  Information  Systems. 

Foreign  Air  Cushion  Vehicles. 

Soviet  Bloc  Capabilities. 

Don't  Know. 

Vaccine  Trials. 

Casualty  Study. 

A  0  Threats  Scenario. 

ACOSS  (Control  of  Space  Structures) 

Assessment  of  Acceptability  of  Digital  Speech  Communication  Systems. 

Is  the  US  Prepared  for  its  next  conflict? 

Guide  for  Transitioning  Army  Missile  Systems  from  Development  to 
Production. 

Electro-Metric  Determination  of  Metals  etc. 

Computer  Technology  Forecast  and  Miliary  Acquistions  System  Process. 
Maintenance  and  Diagnostics. 

Chemical  warfare  in  Central  Europe. 

Fault  Action  of  Equipment. 

Heliopter  Rotor  Down  Wash-Tests. 

Aircraft  Dynamics. 

Tabular  Aids  for  Fitting  Widral  movement  esculant. 

A  Simulation  Model  for  Reliability,  Availability,  Maintainability  (RAMA)  of 
Amphibian  Assault  landing  craft. 
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Chemical  Stimulant. 

IB/Soviet  Strategic  Balance  in  '80-  Can  We  Meet  the  Challenge 
Base  Contracting  Procedural  (2  vols.). 

Operations  Research  -0  Career  Field. 

Human  Dimensions-Protective  Clothing. 

Instructional  Technologoy. 

Truck  Driving. 

Imagery  and  Learning. 

Naval-  Blue-Green  Single  Pole  Downlink  Propagation  Model. 
Corrosion  -  Prevention  in  Tactical  Vehicles. 

Logistics  Support  at  the  Corps  level. 

Potential  for  Conflict  in  Latin  America. 

Jet  Flow  Phenomena . 
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Qjestion:  "*hat  was  the  general  subject  of  the  last  search  of  the  OTIC 
database  you  performed  or  had  performed  for  you?"  (See  page 

^  *  V  •  ) 


Plasticity 
Video  Disk  and  CAI 

Group  on  Forecasting  Methods  for  Defense  Contractors 

Infrared  Transmission  of  Optical  Materials 

EHF  communications 

Moving  Target  Indicators 

Warsaw  Pact  Logistics  Planning  Guide 

Test  Report  on  PLRS  (Position  Locating  and  Reporting  System) 

Undersea  Warfare 

Monopults 

Security  Classification  Management 
Submarine  Simulation 

Russian  Translations  on  Communications  Security 

Underwater  Survival 

Microwave  Electronics 

Command  and  Control  Systems 

Acquisition 

Effects  of  Chemical  Toxicants  on  Airplane  Materials 
Absorption  Phenomena 

Tig  Fusion  Spot  Repair  of  Resistance  of  Defective  Spot  Welds 

Laser  Window  Materials 

Fracture  Mechanics 

Navigation  of  Course  Missiles 

Non-Destructive  Exam  of  Ceramic  Materials 

Flight  Safety 

Computer  Communications  Security 
Navigation  Satellites 

Geophysical  Prospecting  and  Site  Selecting 
High  Altitude. Platforms 
Navy  Transit  System 
Command  and  Control 

Sensitivity  Analyses  of  Platforms  in  Target  Location 

Reflectivity  Measurements  with  10.6  Micrometer  Infrared  Radiation 

Technology  in  Weapons  System 

Towee  Arrays 

Integrated  Optics 

Projectiles 

Shaped  Charges 

Passive  Sonar  Signal  Processing 
Rocket  Propellants 

Compression  of  Solid  Rocket  Propellants 

Nuclear  Propulsion 

Cost  Productivity 

Meteorological  Data  Reduction 

Integrated  Circuits 

Optical  Electronic  Instrumentation 

Miniature  Electronic  Equipment 

CC  Intelligence 

Fluid  Belt  Combustors 

Preparation  of  Radiation-Adhesive  Bonding 
CO2  Lasers 
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Infrared  Detections 

Countermeasures 

Maurick  Warheads  and  Fuses 

Target  Detection 

Lasers 

Defects  of  Airfields 
Electric  Guns 
Lasers 

Tunnelling  Technology 
Landmine  Warfare. 

Radar/Radio  Frequency  Detection 
Archival  Mass  Memory 
Air  Warfare 

Space-born  Detection  System 

Psychological  Impact  of  Retirement  on  Military  Personnel 

Teaching  Methods  and  Training  Devices 

On-line  Searching  of  Bibliographic  Data 

Warsav  Pact  Political  Vulnerability  Study 

Control  Systems 

Laser  Effects 

ADA  (Computer  Language) 

Behind  the  Armored  Debris 

Defense  Repression 

Close  Air  Tactics 

Training  of  Military  Instructors 

Army  War  College  Thesis  (Bibliography) 

Command  Control  Communications  Intelligence 
Mines 

Electromagnetic  Pulse 
Linear  Shaped  Charges 
Mines  in  Urban  Areas 
Underwater  Acoustics 
Soviet  Import  Controls 
Air  to  Air  Missiles 
Diving 

Radar  Cross-Section  Studies 
Synthetic  Radar 
Natural  Language  Processing 
Command  and  Control 
Chemical  Warfare 

Self-Forging  Fragment  Calculation 
Aluminum  Armor  -  Spelling  to  Fragmentation 
Productivity 

New  Vessel  Series  (11)  -  500-T  Class 

White  Phosphorous 

Tactical  Nuclear  Warfare 

Software  Cost  Estimating 

Military  Preparedness  in  the  Middle  East 

War  Gaming  Models 

Item  Identification  -  Personnel  Management 

Ground  Shock  and  Soil  Properties 

Mathematical  Programming 

Logistics  of  the  West  German  Army 

Army  Field  Services 

Contract  Motivation 

Coal  Fuel  Availability 


A-26 


- — 


Tactical  Satellite  Terminals 

Development  of  an  All-Metal ,  Thick  Film,  Cost-Effective  Metallization  for  Solar 
Cells 

Desert  Water  Problems 
Weapon  System  Cost  Growth 
Africa 

Cold  Regions  Engineering 
Gyration 

Determine  Definition  of  Term  "Milestones" 

Productivity  (Contractor) 

Cost  Analysis  Related  to  Production-Engineering 

Management  Information  Systems 

Combat  Maintenance  Capability 

Surfactants  in  Electrochemistry 

Software 

Job  Enrichment 

Built-in  Test;  Automatic  Test  Equipment 

Logistics  Policy  and  Planninc 

Helicopter  Rotor  Downwash 

Rubber  Materials 

Statistical  Sampling 

Reliability 

Chemical  Agent  Simulant 
Strategic  Metals  (Materials) 

Operations  Research  -  Career  Field 
Protective  Clothing 
Instructional  Technology 
Truck  Driving 
Space  Radiators 
Imagery  and  Learning 
Publications  of  Naval  War  College 
Logistics  Support  1982 
Laser  Target  Designations 
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APPENDIX  B 
SURVEY  METHODOLOGY 


The  DTIC  survey  was  conducted  by  telephone .  The  questionnaires 
were  achninistered  to  the  respondents  by  OTIC  interviewers  using  the  inter¬ 
governmental  telephone  network.  All  users  of  OTIC  were  randomly  sampled 
from  one  of  two  sources.  For  those  responding  to  questions  concerning 
technical  reports,  bibliographic  searching,  nanual  searching  and  CAB,  users 
were  sanpled  from  a  list  of  On-Demand  Technical  Report  Orders  for  the 
period  October  1981  through  April  1982.  A  list  of  OTIC  search  requests  was 
used  to  sample  users  of  the  Management  Databanks:  WUIS,  R&DFP  and  IR&D. 
These  users  represented  those  that  requested  OTIC  to  conduct  a  search  for 
them  as  well  as  those  that  performed  the  search  personally. 

Organizations  that  ordered  300  or  more  reports  in  either  hard  copy 
or  microfiche  were  defined  as  large  organizations  and  those  with  orders  of 
less  than  300  reports  for  the  period  were  defined  as  small  organizations 
for  the  purpose  of  this  survey.  In  many  cases  the  organization  had  more 
than  one  location  that  ordered  reports.  If  the  organization  had  at  least 
one  location  with  more  than  300  reports  ordered,  then  that  and  all  other 
locations  of  that  organization  were  defined  as  "large." 

Approximately  80  percent  of  the  technical  reports  users  were 
identified  by  contacting  the  organization  library  or  reports  center,  each 
of  which  provided  names  of  approximately  five  OTIC  technical  reports  users. 
Users  were  then  contacted,  and  80  percent  responded  to  the  questionnaire 
administered.  The  libraries  and  information  cent' js  were  sanpled  from  the 
Technical  reports  On-Demand  requestor  list. 

TWo  main  questionnaires  were  designed  and  adaptations  of  each  were 
used  for  each  group  of  nine  respondents.  The  respondent  groups  were 
characterized  by  whether  they  were  government  or  contractor,  small  or  large 
organizations,  and  by  the  parts  of  the  OTIC  system  they  utilized:  manage¬ 
ment  databanks,  technical  reports,  bibliographic  searches,  TAB,  CAB  and 
combinations  of  the  above. 
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The  users  that  were  sampled  from  lists  of  technical  report  reques¬ 
tors  were  requested  to  report  which  parts  of  the  OTIC  system  they  utilized. 
Based  on  that  the  interviewer  would  ackninister  the  Technical  Reports  Ques¬ 
tionnaire  and  one  other  questionnaire,  either  a  (1)  Bibliographic 
searching,  (2)  TAB,  or  (3)  CAB.  (See  Appendix  C  for  samples  of  question¬ 
naires.)  In  total,  responses  were  received  from  315  individuals. 

The  management  databanks  respondents  were  queried  much  like  the 
technical  reports  respondents.  They  were  divided  according  to  (1)  govern¬ 
ment  and  contractors,  (2)  small  and  large  organizations  and  (3)  the  mnage- 
ment  databank  used.  In  this  case  the  respondents  were  also  divided  between 
those  that  had  requested  OTIC  to  perform  a  search  for  them  and  those  that 
had  searched  OTIC  databases  themselves.  The  databases  covered  in  this 
survey  were:  (1)  R&T  Work  Unit  Information  System  (WUIS)  Data  Base 

describing  research  and  technology  projects  currently  in  progress  at  the 
work  unit  level;  (2)  R&D  Program  Planning  Data  Base  (R&DPP)  describing 
program  planning  at  the  project  and  task  levels  and  (3)  the  Independent 
Research  and  Development  (IR&D)  Data  Bank.  The  respondents  answered 
questions  about  one  specific  search  within  one  databank.  The  number  of 
respondents  to  the  management  databank  questionnaires  was  131. 


Surrey  Instruments 

Two  main  questionnaires  were  designed  for  the  survey  of,  with 
multiple  subquestionnaires  which  were  combined  to  cover  each  respondent's 
usage  of  OTIC. 

For  the  sample  of  on-demand  technical  reports  requestors,  an  intro¬ 
ductory  questionnaire  was  administered  to  identify  the  parts  of  the  OTIC 
system  utilized  (see  Exhibit  A) .  The  users  were  whether  they  had  used 
technical  reports,  management  information  reports,  TAB  and  TAB  indexes,  CAB 
or  performed  bibliographic  searches  in  the  last  year  (summer  1981  -  summer 
1982) .  They  were  asked  to  estimate  monthly  uses,  time  spent  on  those 
individual  uses,  and  the  percentage  of  uses  that  were  classified  and 


EXHIBIT  A 

IDENTIFICATION  OF  DTIC  USAGE 
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limited  (of  technical  reports  only) .  The  OTIC  product  and  service  uses 
reported  acted  as  a  cue  for  interviewers  in  determining  the  subquestion¬ 
naires  to  be  ackninistered. 

Question  2  of  the  introductory  questionnaire  reported  other  sources 
of  information  other  than  OTIC  utilized  by  the  respondent  and  aided  in 
identifying  the  proportion  of  use  conpared  to  use  of  OTIC  (see  Exhibit  B) . 
Question  3  defined  the  work  role  of  the  respondents,  which  was  important 
later  in  the  analysis  of  the  survey  results. 

The  subquestionnaires  focused  on  the  respond  ait's  last  use  of  OTIC 
product  or  services  and  the  characteristics  and  value  of  the  information 
gathered  through  that  request.  In  addition,  the  users  were  queried  con¬ 
cerning  the  amount  of  time  spent  reading  or  searching,  the  source  of  physi¬ 
cal  access,  and  how  the  data  gathered  were  applied  to  projects  or  tasks. 

The  possible  combinations  of  the  questionnaire  were  numerous  and 
acted  to  define  the  respondents  by  their  types  of  use  of  OTIC  information . 
They  also  helped  to  link  the  types  of  use  to  the  user's  perceptions  of  the 
value  of  the  data  gathered  using  the  OTIC  products  and  services. 
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EXHIBIT  B 

OTHER  SOURCES  OF  INFORMATION  AND  WORK  ROLES  IDENTIFICATION 
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APPENDIX  C 


DTIC  SURVEY  QUESTIONNAIRES 


interviewer  name 
date  of  interview: 


name: 


organization : 


V.  MANAGEMENT  DATA  BANK  SURVEY 


phone : (  ) 


According  to  our  records, 

1)  You  searched 

2)  DTIC  searched  for  you 


from 

a)  the  R&T  Work  Unit  Data  Bank 

b)  the  R&D  Program  Planning  Data  Bank 

c)  the  Independent  R&D  Data  Bank 


r  within  the  last  month.  Could  you  please  give  us  the  name  and  telephone  I 
numer  of  _  individuals  for  whom  requests  were  made?  _ J 
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L  \t 


INTRODUCTION 


Hello,  nay  I  speak  to 


(Name  of  person  called) 


Hello,  this  is  (name  of  interviewer) .  I  am  with  the  Department  of  Defense 
Technical  Information  Center.  We  are  making  some  inquiries  in  order  to 
gather  data  concerning  distribution  of  our  products  and  services  to  libraries  and 
information  centers. 

I  would  appreciate  it  if  you  would  take  a  moment  to  answer  several  questions 
concerning  this  inquiry.  Any  information  will  be  kept  in  strictest  confidence 
(Defense  Department  #3507) . 


We  understand  that  you  recently  received  output  from  a  search  of 

a)  R&D  Program  Planning  Data  Bank  (R&DPP) 

b)  R&T  Work  Unit  Information  System  (WUIS)  Data  Bank 

or  c)  Independent  Research  and  Development  (IR&D)  Data  Bank 
The  questions  I  am  going  to  ask  you  are  about  that  search  output. 

1.  For  what  purpose  did  you  request  the  search? _ 


2.  Were  the  results  useful  for  that  purpose?  Yes 


3.  To  the  best  of  your  knowledge,  is  the  information  you  obtained: 


complete 


Don't  Know 


4.  Did  you  pass  the  information  you  obtained  on  to  anyone  else? 


How  many  people?_ 
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5.  Has  Che  information  you  obtained  from  the  data  bank  saved  you  and/or  your 
covorkers  any  time? 


No  □ 

Don't  Know  □ 

6.  If  we  consider  the  average  hourly  wages  of  your  staff  workers  times  the  hours 
of  search  time,  what  is  the  approximate  dollar  value  of  the  time  you  and/or 
your  co-workers  saved?  (Insert  zero  "0"  if  none.) 


7.  What,  if  any,  were  the  dollar  savings  for  other  things  such  as 
equipment  and  supply  costs?  (Insert  zero  "0"  if  none.) 


8.  Could  you  describe  how  it  saved  you  time? 


(Probe:  What  would  you  have  had  to  do  if  you  hadn't  read  that  particular 
information?) 


9.  About  how  many  times  in  the  past  have  you  used  DTIC's  management  data  bases? 


10.  How  frequently  do  you  expect  to  use  the  system  in  the  future? 


H  M 


■hum  mi  i  j 


NAME: 

ORG: 


DTIC  I'SER  QUESTIONNAIRE _ 

TITLE: 

INTRODUCTION  PHONE: - 

Hello,  may  1  speak  to  _ _? 

(Name  of  person  called) 

Hello,  this  is  (name  of  interviewer).  I  am  with  the  Department  of  Defense 
Technical  Information  Center.  We  are  making  some  inquiries  in  order  to 
gather  data  concerning  distribution  of  our  products  and  services  to  libraries  and 
information  centers. 

I  would  appreciate  it  if  you  would  take  a  moment  to  answer  several  questions 
concerning  this  inquiry.  Any  information  will  be  kept  in  strictest  confidence 
(Defense  Department  #3507). 

1.  We  are  Interested  in  finding  out  about  your  use  of  defense  related  publications, 
particularly  those  distributed  by  DOD's  Technical  Information  Center.  I  am 
going  to  read  you  a  list  of  six  products  and  services  provided  by  DTIC.  For 
each  one,  I  wcjld  like  to  know:  (1)  whether  you  have  used  the  product  or  service 
in  the  last  year,  (2)  if  yes,  how  many  times  you  use  the  product  monthly  or  yearly, 
(3)  the  number  of  hours  you  spend  using  the  product  or  service  in  an  average 
month  or  year,  and  (4)what  proportion  of  this  use  is  relative  to  classified 
materials  and  what  proportion  is  relative  to  limited  materials. (FOR  CATEGORIES  a.-b.) 


Product  or  Service 

Used  in 
Last  Year 
Yes  No 

Average  Monthly/ 
Yearly  Uses 
(Specifv) 

Time  Spent  Using  Percent 
(Specify  minutes  Classif 
or  Hours) 

Percent 

limited 

"a.  Technical  Reports  in 

Paper  Copy 

□  □ 

(Mo/Yr) 

(Hrs/nin)  1 

X 

1. 

■b.  Technical  Reports  in 
Microf iche 

n  a 

(Mc'Yr) 

(Hrs/min) 

• 

V. 

c.  Management  Information 
Reports 

□  □ 

(Mo/Yr) 

(Hrs/min) 

pd.  TAB  and  TAB  Indexes 

□  □ 

(Mo/Yr) 

(Hrs/nin) 

yes,, 

:  2+,  1 

en  ♦  | 

Le.  CAB  (Current  Awareness 
|  Bibliographies) 

.  f.  Bibliographic  Searching 

□  □ 

(Mo/Yr) 

(Hrs/min) 

□  □ 
C=J  □ 

(Mo/Yr) 

(Hrs/min) 

IV 

fl.  By  your  organization 

f 2 .  Bv  DTIC 

(Mo/Yr) 

(Hrs/min) 

□  □ 

(Mo/Yr) 

(Hrs/nin) 

(IF  RESPONDENT  ANSWERS  NO  TO  (a),  (b),  (d),  (e),  and  (f),  STOP  AFTER  Question  3.) 

Can  You  tell  us  how  many  researchers  (scientists  and  engineers)  are  in  your 
organizaticn.7 
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We  are  also  interested  in  your  use  of  other  sources  of  information. 

Please  indicate  your  average  monthly  use  and  the  time  spent  on  the 
following: 

Average 

Monthly 

Uses  Time  Spent  Using/Month 

a.  Non-DOD  technical  reports  _  _ mins./hrs. 


b.  Journal  articles  _  mins./hrs. 


c.  Other  materials  _  _ mins./hrs. 


What  is  your  primary  work  role  within  the  organization,  that  is,  what  do 
you  spend  the  largest  proportion  of  your  time  doing? 

a.  Management  |  | 

b .  Teaching  j  ] 

c.  Research  |  1 

d.  Other  I  I 


I.  TECHNICAL  REPORT  QUESTIONNAIRE 


1.  We  are  interested  in  your  use  of  reports  provided  by  DTIC.  Can  you  tell  us 
what  report  you  read  last? 


(SPECIFY:  FICHE _ or  PAPER  COPY _  AND  CLASSIFIED _ UNCLASSIFIED/LIMITED 

or  UNCLASSIFIED/UNLIMITED _ ) 

2.  How  did  you  initially  find  out  about  this  technical  report? 

It  was  routed  to  me .  1 

From  another  person  (i.e.,  a  colleague) .  2 

Cited  in  another  report  or  journal  article .  3 

From  an  announcement  publication  which  include: .  A 

STAR _  TAB _  GRA&I _  OTHER  (name) _ 

From  CAB  (Current  Awareness  Bibliography) .  5 

From  a  Bibliographic  Search .  6 

Done  by  your  organization  _ 

Done  by  DTIC  _ 

Other  (describe) _ . . .  7 

3.  From  which  source  did  you  obtain  physical  access  to  this  technical  report? 

It  was  sent  to  me  via  standard  distribution .  1 

It  was  sent  to  my  Library/organization  via  standard 

distribution .  2 

I  ordered  it .  3 

My  Library/organization  ordered  it, .  A 

Other _  . . .  5 

A.  How  thoroughly  did  you  read  this  report? 

Kith  great  care.. .  1 

With  attention  to  the  main  points .  2 

Just  to  get  the  idea .  3 

5.  To  how  many  others  did  you  recommend  this  report? _ 


C-6 


In  which  of  the  following  ways  have  you  used,  or  do 

you  plan 

to 

use,  the  last 

report  you  read? 

Yes 

No 

Don ' t  Know 

To  apply  its  findings  to  a  current  project . 

1 

2 

3 

To  apply  its  methodology  to  a  current  project... 

1 

2 

3 

In  preparation  of  a  research  proposal . 

1 

2 

3 

In  preparation  of  an  article,  book,  review,  or 
report . 

1 

2 

3 

As  a  citation  in  a  journal  article,  book,  review 
or  report . 

1 

2 

3 

For  professional  development,  current  awareness, 
or  general  interest . 

1 

2 

3 

In  preparation  of  a  lecture  or  presentation . 

1 

2 

3 

For  the  planning,  budgeting  and  management  of 
research . 

1 

2 

3 

No  use . 

1 

2 

3 

Other  (describe)  .... 

1 

2 

3 

7a.  Has  reading  these  reports  saved  you  and/or  your  co-workers  any  time  on  any  current 


task  or  project? 

Yes .  1 

No .  2 

Don '  t  Know .  3 


7b.  What  did  you  and  your  staff  save  as  a  result  of  that  reading?  _ 

8.  If  you  consider  the  amount  of  staff  time  devoted  to  searching  for  these 
materials  and  the  average  salary  of  the  staff,  what  is  the  approximate 
dollar  value  of  the  time  you  and/or  your  co-workers  saved? 

(Insert  zero  "0"  if  none.)  _ 

9.  What,  if  any,  were  the  dollar  savings  for  other  things  such 

as  equipment  and  supply  costs?  (Insert  zero  "0"  if  none.)  _ 


10.  Could  you  describe  how  it  saved  you  time? 


(Probe:  What  would  you  have  had  to  do  if  you  hadn't  read  that  particular 
information?) 
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II.  MANUAL  SEARCH  QUESTIONNAIRE 


1.  When  was  the  last  time  you  used  TAB?  _ _ 

What  was  it  for:  Browsing g  Problem-related  search □ 

2.  Did  you  personally  perform  the  search,  or  was  it  performed  for  you? 

Performed  search  personally  □ 

Search  was  performed  by  someone  else  □  Who?  _ 


3.  How  much  time  did  you  spend  on  the  search? _ 

4a.  How  many  items  were  identified? _ 

b.  How  many  were:  classified  1  1 

unclassified/  limited|  | 

unclassified/unlimited  1  | 

5.  To  how  many  of  these  items  did  you  physically  obtain  access  (or  intend  to 

obtain  access) ? 


6a.  How  many  items  did  you  read  (scan  beyond  the  title  page)?  _ 

b.  How  many  of  these  items  read  were: 

Classified 

Unclassified /Limited  _ 

Unclassified/Unlimited  _ 

7a.  Has  reading  these  reports  saved  you  and/or  your  co-workers  any  time  on  any 
current  task  or  project? 


Yes .  1 

No .  2 

Don '  t  Know .  3 


7b,  What  did  your  staff  save  as  a  result  of  that  reading? 
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8.  Considering  only  direct  salaries,  what  is  the  approximate  dollar 
value  of  the  time  you  and/or  your  co-workers  saved?  (Insert 
zero  "0"  if  none.) 


9.  What,  if  any,  were  the  dollar  savings  for  other  things  such 
as  equipment  and  supply  costs?  (Insert  zero  "0”  if  none.) 


10.  Could  you  describe  how  it  saved  you  time? 


(Probe:  What  would  you  have  had  to  do  if  you  hadn't  read  that  particular 
information?) 
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III.  BIBLIOGRAPHIC  SEARCH  QUESTIONNAIRE 


1.  What  was  the  general  subject  of  the  last  search  of  the  DTIC  data  base  you 
performed  or  had  performed  for  you? 

2.  What  method  of  searching  was  used? 

Search  was  performed 
by  your  organization  _ 

DTIC-Provided 

Searching  _ 

3.  Did  you  personally  perform  the  search,  or  was  it  performed  for  you? 

Performed  search  personally 

Search  was  performed  by  someone  else  By  Whom? _ 

A.  How  much  time  did  you  spend: 

Requesting  and/or  conducting  search  _ 

Reviewing  search  output  _ _ 

5.  How  many  citations  did  you  receive  as  output  from  the  search?  _ 

6a. How  many  citations  were  relevant? _ 

6b. How  many  were:  classified _ unclassif ied/limited _ unclassif ied/unlimited 

7.  To  how  many  of  these  items  did  you  actually  obtain  access? 

8a. How  many  items  did  you  read? 
b.How  many  items  read  were: 

Classified  _ 

Unclassified/Limited  _ 

Unclassif ied/Unlimi ted _ 

9.  Has  reading  these  reports  saved  you  and/or  your  co-workers  any  time  on  any 
current  task  or  project? 


Yes .  1 

No .  2 

Don '  t  Know .  3 
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10.  If  we  consider  average  salary  of  your  staff  workers  times  the  amount  of 
time  they  would  spend  searching,  what  is  the  approximate  dollar  value 
of  the  time  you  and/or  your  co-workers  saved?  (Insert  zero  "0"  if  none.) 


11.  What,  if  any,  were  the  dollar  savings  for  other  things  such  as 
equipment  and  supply  costs?  (Insert  zero  "0"  if  none.) 


12.  Could  you  describe  how  it  saved  you  time? 


(Probe:  What  would  you  have  had  to  do  if  you  hadn't  read  that  particular 
information?) 
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IV.  CURRENT  AWARENESS  BIBLIOGRAPHY  QUESTIONNAIRE 


1.  When  did  you  last  review  a  current  awareness  bibliography? 


Haw  many- were j  classified-  unclassified/Limited _ unclassified /unlimited 

2.  How  many  relevant  citations  did  you  identify? 


3.  How  many  reports  did  you  request  or  plan  to  get?  _ 

4.  How  many  items  did  you  read?  _ 

5.  How  many  of  the  reports  requested  were: 

Classified  _______ 

Unclassified/Limited  _ 

Unclassified/Unlimited  _ 

6.  Has  reading  these  reports  saved  you  and/or  your  co-workers  any  time  on  any 
current  task  or  project? 


Yes .  1 

No .  2 

Don't  Know .  3 


7.  If  we  consider  the  average  staff  worker's  salary  and  the  time  saved  x  number 
of  staff,  what  is  the  approximate  dollar  value  of  the  time  you  and/or  your 
co-workers  saved?  (Insert  zero  "0"  if  none.) 

$ 


8.  What,  if  any,  were  the  dollar  savings  for  other  things  such  as  equipment 
and  supply  costs?  (Insert  zero  "0"  if  none.) 


9.  Could  you  describe  how  it  saved  you  time? 


(Probe:  What  would  you  have  had  to  do  if  you  hadn't  read  this  particular 
information?) 
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online  searching  of  the  Technical  Reports  (TR)  data  base.  Current  Awareness  Bibliographies 
(CAB),  and  Technical  Abstract  Bulletins  (TAB);  and  provision  of  management  information  froi^ 
three  management  data  banks.  The  amount  and  kinds  of  use  of  each  of  these  products  and 
services  is  dddressed,  as  is  the  value  associated  with  use. 

Study  findings  establ Lsh  significant  levels  of  value  associated  with  the  provisj^ins  of 
DTIC  products  and  services.  There  are  substantial  readings  of  DoD  technical  reports,  with 
about  12. A  million  such  readings  estimated  for  1982.  The  savings  value  associated  with  an 
individual  reading  —  $4,600  on  the  average  —  is  high.  The  total  value  to  the  Department 
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■>t  Defense,  reflecting  only  those  readings  coming  through  DTIC  bibliographic  products, 
is  $37.5  billion.  This  suggests  a  considerable  return  on  the  investment  in  DTIC's 
information  products  and  services. 


